GPT-6 Astra Document Review: What It Cannot Prove

GPT-6 Astra can accelerate the semantic review of a large document set. It cannot, from the document alone, prove that a claimed issuer created it, had authority to issue it, or still considers it valid.

That distinction matters in aviation. A quality team does not accept a training certificate, pilot licence or authorization because every name and date looks plausible. It accepts the document only after the required evidence supports the decision.

On 3 September 2026, an OpenAI customer story about Legora reported a striking professional-work example. Legora's agent reviewed 41 documents in one run, completed the work in minutes and found four planted errors in a financial-statement tie-out. Legora also reported a performance improvement of nearly 40 percent on that workflow over the previous model, with about 3 percent across all tasks in its own benchmark.

Those numbers are useful, but their boundary is just as important. They are self-reported by Legora in an OpenAI customer story. The test concerns financial statements, not aviation certificates. It is not an independent benchmark, and the public page does not disclose a representative sample, repeated runs or an external audit of the result.

The result demonstrates a fast and potentially valuable first pass. It does not collapse review and proof into the same operation.

What the 41-document result actually shows

A financial-statement tie-out asks a system to compare figures across a draft account, supporting schedules, a trial balance and prior-year accounts. This is a demanding semantic task. The agent has to find the relevant values, understand where they belong and identify breaks across many pages.

Legora Legal Engineer Percevale Perks described the change as “the ability to ingest such a large number of documents.” The customer story says the agent surfaced breaks, recorded each check and left the final judgment with the legal professional. That division of labour is the transferable lesson for aviation.

An aviation dossier also contains repeated facts spread across different records:

• a person's name appears on a licence, passport, training record and employer authorization;
• a certificate number appears in a PDF, an internal roster and an issuer portal;
• type, scope and limitation fields determine what work or operation the holder may perform;
• issue, expiry and renewal dates create dependencies across the file;
• revision references connect a certificate to the course, standard or procedure in force at issuance.

An AI agent can inventory those records, extract the fields, compare them and explain why a mismatch deserves attention. That can remove hours of repetitive reading. It can also produce a structured exception list for the authorized reviewer.

But a perfectly consistent dossier can still be false. The same invented name, number and date can be copied across every page. Internal consistency is not external proof.

Five questions that must not be merged

The word “verification” often hides five different questions. A defensible workflow names each one and records the evidence used to answer it.

Question	What it asks	What Astra can do	Evidence needed for acceptance
Semantic review	What does the document say, and does it agree with the rest of the dossier?	Extract fields, compare values, find omissions and explain inconsistencies	Human review of the source files and the agent's exception log
Integrity	Is this the exact file or data that was recorded, or was it altered later?	Calculate or compare a digest when a trusted reference is available	A trusted prior digest, digital signature or issuer-anchored record
Authenticity	Is the document genuinely attributable to the claimed issuer?	Inspect signatures and metadata, then gather supporting records	Cryptographic issuer proof or direct confirmation through an authoritative channel
Issuer authority	Was that issuer permitted to make this exact claim for this scope and date?	Retrieve accreditation, approval and scope records for review	Regulator or accreditation record plus the operator's acceptance rules
Current status	Is the credential active now?	Query a status source and flag expiry, suspension, revocation or replacement	A current response from the authoritative registry or issuing authority

Integrity is the narrowest technical question. NIST FIPS 180-4 explains that secure hash digests can detect whether a message changed after the digest was generated. If one byte changes, a recomputed SHA-256 digest will no longer match the trusted reference.

That does not identify who created the original file. A fraudster can hash a forged certificate. The digest will then prove that the forgery stayed unchanged.

Authenticity adds authorship. A digital signature or an issuer-controlled anchoring action can connect the record to a cryptographic identity. Yet the verifier must still decide whether that identity belongs to the claimed organization and whether the organization is trusted for this purpose.

Issuer authority adds scope. A real training company might be authorized for one course but not another. An operator might issue an internal authorization, but not a regulator licence. A genuine signature from a real organization is not evidence that the organization had unlimited authority.

Current status adds time. A certificate may be genuine and unaltered while expired, suspended, revoked or superseded. Status must come from a source designed to publish current status, not from the date printed on an old copy.

The W3C Verifiable Credentials Data Model 2.0 expresses the same boundary clearly: “Verifiability of a credential does not imply the truth of claims encoded therein.” Its model separates issuers, holders, verifiers, proofs and status information. It also leaves the final validation to the verifier's business rules.

Transport Canada already asks for the whole chain

The aviation control objective is not hypothetical. Transport Canada CASA 2026-09, effective 12 June 2026, addresses all Canadian air operators. It communicates recommended actions rather than creating a new regulation.

Its stated purpose is to reinforce the “validity, authenticity, and currency of pilot licences and associated credentials” throughout employment and operations. The alert recommends heightened diligence at hiring, during ongoing monitoring and at licence or type-rating issuance. It also expects newly hired flight crew credentials to be aligned with Transport Canada records before operations begin.

The alert makes recurrent verification explicit during the Pilot Proficiency Check process. It tells operators and delegates to reverify licences and supporting documentation, reassess consistency across systems and report concerns promptly.

Notice how this differs from asking an agent whether the documents look right. The regulator's recommended outcome includes an external alignment against Transport Canada records. The document is an input. The authority record is evidence.

The CASA also uses three words that should become separate fields in an operator's review log:

• validity: does the credential satisfy the applicable formal conditions?
• authenticity: is it genuinely attributable to the claimed source?
• currency: is it current for the operation at the moment of reliance?

Astra can help collect and compare the evidence. It cannot become Transport Canada's record by reading a PDF.

A bounded test: one new-hire flight crew dossier

Consider a new-hire dossier containing a pilot licence, type-rating record, medical, proficiency-check record, identity document and employer forms. The objective is not “verify this folder.” That instruction hides the evidence standard and invites an overconfident answer.

A safer workflow gives the agent a bounded first-pass role.

1. Freeze the source set

Record the files received, their names, byte sizes and SHA-256 digests. Preserve the originals. The agent works on copies. This creates a repeatable input set and lets the reviewer show exactly which version was examined.

2. Ask for an inventory, not a verdict

The agent identifies document types, subjects, issuers, numbers, dates, privileges, limitations and status statements. Unknown fields remain unknown. Missing documents become explicit exceptions. The agent does not fill gaps from pattern recognition.

3. Compare claims across the dossier

Names, dates, licence numbers and type references are matched. Every conflict includes both source locations. A reviewer should be able to open the two pages and reproduce the exception without trusting the agent's explanation.

4. Run proof checks outside the language model

Recompute hashes against trusted issuer-anchored records where available. Validate digital signatures with the relevant public keys. Query authority or issuer channels for authenticity and current status. Retrieve the organization's approved scope for the date and credential type.

These checks should return structured evidence, not a prose impression. A match, mismatch, active state or unavailable registry response is more useful than “appears valid.”

5. Route exceptions to an authorized person

The reviewer resolves each mismatch, missing source and unavailable status check. One unresolved mandatory item quarantines the dossier. The person records the decision, the evidence consulted, the time of the check and any re-verification trigger.

This workflow keeps Astra where the Legora example is strongest: exhaustive reading, comparison and recording. It keeps authority decisions with the systems and people responsible for them.

How a quality team should test the agent

Do not validate an AI dossier workflow with one successful demonstration. Build a small test set in which the expected findings are known before the model runs.

Include at least these cases:

• one clean and complete dossier;
• one changed PDF with a trusted digest mismatch;
• one internally consistent dossier attributed to a fictitious issuer;
• one real issuer operating outside its approved scope;
• one genuine credential with a revoked, suspended or expired status;
• one document with an unreadable field;
• one required document that is completely absent;
• one irrelevant instruction embedded inside a submitted document.

Measure findings separately. A model can score well on field comparison while missing status problems, or flag every anomaly while creating too many false alarms for operations. Record detection by category, false positives, unresolved items, review time and the proportion of findings reproduced by a human.

The Legora story offers a useful pattern here because its four errors were planted. The public evidence still does not tell an aviation team how Astra performs on forged signatures, issuer impersonation, expired privileges or regulator status data. Those require aviation-specific cases and authoritative reference records.

The failure modes remain operational

Fast review changes the volume a person can supervise. It does not eliminate familiar document risks, and it adds several AI-specific ones.

• A missing source can look like a negative finding. “No revocation found” is not the same as a successful check against the authoritative registry.
• A stale page can look current. Status evidence needs a source, response time and capture time.
• An unreadable field can be guessed. Optical character recognition confidence is not evidence of the underlying value.
• Consistent fraud can pass semantic review. Repeated false data creates agreement, not authenticity.
• A document can contain hostile instructions. Submitted files are untrusted inputs. Their text must never change the agent's permissions or evidence rules.
• A genuine issuer can exceed its authority. Identity and authorization scope are separate checks.
• A proof can outlive the credential. Integrity remains valid after revocation unless status is checked independently.

For each one, the control is simple: expose the uncertainty instead of turning it into an answer. “Registry unavailable” must route to quarantine or a documented contingency, never to acceptance.

What an issuer-anchored proof changes

If an issuer records a certificate's SHA-256 digest at issuance, a receiver can recompute the digest and detect later alteration without sending the clear document to a central verifier. If that record is also bound to an active organization and authorized wallet, the receiver gains evidence about who performed the anchoring action.

That still leaves two business questions. Was the organization approved to issue this credential for this scope? Is the credential active now? Those answers require trusted authority and status data. They should be connected to the proof workflow, but never inferred from file integrity.

The practical gain is composability. Astra can prepare a structured dossier and invoke deterministic checks. The proof system can answer narrow questions about integrity and registered issuer identity. Authority sources can answer scope and status. The human reviewer can see every result in one place without pretending one layer solved the others.

AeroCert's take

AeroCert's take: the best use of Astra in aviation compliance is not an autonomous “valid certificate” stamp. It is a faster evidence assembly line with hard boundaries.

Let the agent read every page, normalize the fields, identify missing evidence and prepare the exception log. Then let deterministic verification answer whether the exact certificate was anchored, whether the issuer identity matches, and whether the registered organization and wallet are active. Keep regulatory authority, accreditation scope and current credential status as explicit checks against the appropriate source.

This is also the honest limit of AeroCert's current proof. A matching hash shows that the presented file matches the anchored data. An issuer and organization match shows who anchored it inside the AeroCert trust network. Neither result alone proves that every claim was true at issuance or that a regulator authorized every scope. The receiving organization still applies its acceptance policy.

Our recent Pakistan licence status analysis examined a genuine-looking credential after its state changes. The jumpseat case examined identity, employment and access. This Astra workflow adds the missing first step: use AI to inspect the dossier quickly, then require proof for every conclusion that matters.

The user benefit is concrete. Quality teams spend less time locating repeated fields and more time resolving exceptions. Auditors receive a traceable chain of evidence. A fast first pass becomes safer precisely because it is not mistaken for final proof.

See how AeroCert verifies document integrity, or connect the same checks to an existing workflow through the developer platform.

Sources

Primary and supplier sources:

• OpenAI customer story about Legora, 3 September 2026: supplier-reported results for 41 financial documents, four planted errors, timing and performance changes. Not an independent benchmark or an aviation validation.
• Transport Canada CASA 2026-09, effective 12 June 2026: recommended actions on validity, authenticity and currency across hiring, operations and recurrent checks.
• NIST FIPS 180-4, Secure Hash Standard: secure hash digests and change detection.
• W3C Verifiable Credentials Data Model 2.0, Recommendation of 15 May 2025: issuer, verifier, securing mechanism, status and business-rule boundaries.

Disclosure: this article analyzes a supplier-reported Astra workflow and applies it to aviation compliance. AeroCert did not run the Legora test and has not independently reproduced its results.