How to detect an AI-generated PDF

Which checks give an answer that is simply true or false, which ones give a probability, and exactly what each of them can and cannot establish.

Two questions hide inside “was this PDF made by AI”. One asks whether the file's signed statement about its own origin still holds. A yes or a no. The other is a score, and a score has an error rate.

Almost every confident claim about AI document detection quietly moves an answer from the second kind into the first. A deterministic check has two outcomes and no error bar; a probabilistic one returns a number, and a number needs a measured false-positive rate before anyone can act on it. Only one is safe to state as a fact.

Everything above the last box is a way of deciding how urgently to do the last box.
the file in front of you 1 · DETERMINISTIC Does the manifest validate? Does the check digit match? Is there a second revision? What does the signature cover? Yes or no. No threshold to tune. 2 · PROBABILISTIC “Does this page look AI-made?” AUC 0.563 on AI-forged documents (0.98 in-distribution · 0.50 is chance) Queue ordering only. Never evidence. go to the issuer

1. The deterministic layer: C2PA provenance

C2PA is the only check on this page with no threshold to tune. A manifest is embedded in the file and signed by whoever produced it; its hard binding is a SHA-256 hash over the bytes. Recompute, compare, and it validates or it does not. Specification and reference implementation are both public.

What a valid manifest proves, and it is a short list:

  • The file was processed by the holder of that signing key, at the time recorded.
  • The bytes have not changed since. Any change at all breaks the hash.

What it never proves:

  • Authorship. A signer is whatever pipeline handled the file last, not the person who wrote the content.
  • Truth. A valid manifest on an invoice says the file is the file that was signed. It says nothing about whether the invoice was ever issued, owed or paid.
  • Trust. Our self-signed fixtures return validation_state: Valid and signingCredential.untrusted at once. The mathematics holds, the chain is unknown. Software that reads one field and reports the other is where wrong sentences come from.

An absent manifest means nothing at all

We measured five ordinary, C2PA-unaware re-saves of a signed image: a JPEG re-encode at quality 70, a PNG re-save, a PNG-to-JPEG conversion, a resize and a screenshot. All five destroyed the credential completely, with no markers left in the file. Stripping provenance is not an attack; it is a side effect of saving a file.

When a manifest fails, say what failed

The correct sentence is “this file was modified after it was signed”, or “the signed claim itself no longer matches”. Our fixtures produce both codes: assertion.dataHash.mismatch when the content changed under an intact claim, claimSignature.mismatch when the claim bytes changed. Neither is the word “tampered”, and neither is a finding of fraud.

A manifest can break in two places, and each one is a different sentence.
the claim who signed it, when, with what tool a SHA-256 of the asset bytes the page, the picture, the file itself SHA-256 change the claim change the bytes claimSignature.mismatch the signed statement no longer matches assertion.dataHash.mismatch the content moved under an intact claim Both read out as: modified after signing. signingCredential.untrusted A property of the certificate chain, orthogonal to both. Valid and untrusted arrive together.

Image vendors emit this; nobody emits it for documents

Anthropic announced on 11 August 2026 that Claude signs generated .svg, .png and .jpg, noting the metadata “can disappear during format conversions, screenshots, or re-saving”. OpenAI has shipped Content Credentials since 2024 and now pairs them with SynthID; Google embeds SynthID at generation. As of August 2026 no major provider attaches a manifest to a generated PDF, DOCX or spreadsheet.

The reference implementation reads a C2PA manifest out of a PDF and cannot write one in: the only format in its table with that asymmetry.
c2pa-rs READS c2pa-rs WRITES WHO SIGNS THIS TODAY JPEG yes yes PNG yes yes SVG yes yes OpenAI, Anthropic, Google image output PDF yes no nobody, for generated documents

Support table at contentauth/c2pa-rs, read 14 August 2026. Read-only means a PDF can carry a credential it was given, and the standard toolchain will not put one there.

So provenance in documents is a trust-elevating check: its presence is a strong positive signal, and its absence is the overwhelmingly normal case.

2. What we measured: a signature inside a PDF

A PDF is a container, and its embedded images are separate files with their own provenance surface. Nobody measured whether a signed image keeps its credential once placed into a document. We signed one, put it through every PDF generator on one laptop, pulled the image streams back out and re-validated each.

  • 6/6JPEG paths kept the credential, byte-identically
  • 0/2PNG paths kept it
  • 9/9PDFs read as “No claim found”
A signature can survive being put into a PDF while the standard tool for reading signatures calls that PDF unsigned.
signed JPG manifest valid Chrome · img2pdf Ghostscript · qpdf PDF extract the stream → Valid · 6 of 6 signed PNG manifest valid img2pdf · Chrome PDF recompressed raster → lost · 0 of 2 THE BLIND SPOT $ c2patool chrome-jpg.pdf Error: No claim found All nine PDFs answer this, including the six whose embedded JPEG holds a fully valid manifest. The reader looks at the document level; the credential is one deeper.
Path into the PDF Source format Result
img2pdf JPEG Stream byte-identical, manifest still valid
Chrome print-to-PDF (natural size and scaled) JPEG Stream byte-identical, manifest still valid
Ghostscript (direct and as a re-pass) JPEG Stream byte-identical, manifest still valid
qpdf --linearize JPEG Stream byte-identical, manifest still valid
img2pdf, Chrome print-to-PDF PNG Recompressed to a raw raster: credential gone
Chrome print-to-PDF SVG Drawn as vector, no image stream at all: nothing to carry

The format decides, not the generator: a PDF has no PNG container, so those pixels are re-emitted and the chunk holding the credential has nowhere to go. Of the three formats Anthropic signs, exactly one reaches a document, and it is the one document workflows use. Full method and tool versions: does a C2PA signature survive a PDF?

What Tamperlens does with C2PA today, stated exactly. The image engine detects the presence of a Content Credentials container in a standalone image (the JUMBF box in a JPEG, the caBX chunk in a PNG), and reports it as an informational signal that says, in the report itself, “presence detected, manifest not parsed”. It does not decode the manifest and does not verify the signature chain. It does not read C2PA inside a PDF at all; that is proposed work, not a shipped feature. Everything in the two sections above is the standard, or our own measurement with the official command-line tool.

3. Structure signals the checker reads today

These are deterministic reads of facts whose meaning is probabilistic. Whether the string is in the file is not a matter of opinion; what it implies about the document usually is. Keeping those apart is most of the discipline.

A full rewrite regenerates every signal in the top row. The arithmetic in the bottom band is the only layer that survives it.
WHERE EACH SIGNAL LIVES %PDF-1.x version body objects pages, fonts, images, streams xref object offsets trailer /Info /ID document metadata producer era font subsets revision count Info vs XMP page-content edits the /ID pair A FULL REWRITE ERASES ALL OF IT Load and re-save with almost any PDF library and every signal above is regenerated as if the file were new. AND THIS IS WHAT IS LEFT Balance chains, totals, check digits. Arithmetic is a property of the claims printed on the page, not of the container.
Signal What is read, exactly Why it is not a verdict
Producer and creator Info dictionary and XMP name the software that wrote the bytes: ReportLab PDF Library, Skia/PDF m### for headless Chrome, version strings for LibreOffice, plain names for WeasyPrint and wkhtmltopdf Plain-text fields. Anyone who can write a PDF can write Adobe PDF Library into them: read it as an outlier (producer lookup)
Generator metadata in embedded images IPTC trainedAlgorithmicMedia in XMP; an AUTOMATIC1111 parameters block; a ComfyUI prompt or workflow graph. A file carrying a generation-parameter block was not photographed The one image family that reaches high severity, but only in the presence direction. Absence is still nothing
Incremental updates and revisions PDF grows by appending, so a file can literally contain its own earlier versions. Counting them is exact Signing, filling a form and adding an annotation are all routine causes (incremental vs full rewrite)
Signature coverage The /ByteRange names the exact bytes protected; comparing its last byte to the file's last byte is arithmetic Answers only whether something was appended after signing: not whether the appended thing was hostile

A document generated whole: one revision, coherent metadata, one producer, never opened in an editor. Passes every structural check above, and that is precisely the shape of a machine-written document. Structure catches editing. For a file born complete, the load moves to the next section.

4. Arithmetic: the strongest surviving content signal

The most reliable tell on model-generated financial documents is not visual at all. In the GPT4o-Receipt study (1,235 model-generated receipts against authentic ones), humans spotted visual artefacts better than the machines did and were still worse at telling AI-made from real. The signal that worked was the one nobody could see: the numbers did not add up.

A running balance is a chain: every row constrains the next, so a single edited transaction breaks the column unless the forger re-derives all of it.
line amount balance opening 1,000.00 transfer in +250.00 1,250.00 payment −90.00 1,110.00 payment −110.00 1,000.00 ✓ ✓ ✗ ✗ Change one amount and every balance below it contradicts the document itself, no reference copy needed. Same shape for a check digit: a boleto's printed line, a PIX CRC, an NF-e access key.

Note the asymmetry, because it is the honest half. A failed check digit is evidence; a passing one is not. Any generator that implements the rule correctly produces figures that pass. And our engine only reports a broken balance chain after the column has proved it is a balance, several links holding in sequence, because most number columns in real documents are not balances at all.

5. The probabilistic layer and its real numbers

Detectors trained on hand-made tampering fall to near chance on diffusion-made tampering. Four independent 2026 papers measured it. Chance is an AUC of 0.50.

Every detector that scores 0.96: 0.98 on hand-made tampering lands between 0.50 and 0.60 on diffusion-made tampering.
AUC · 0.50 IS CHANCE NEAR-CHANCE BAND humans, on inpainted documents 0.501 general model, zero-shot judge 0.509 the generating model judging itself 0.532 DocTamper, AI-forged documents 0.563 0.98 in-distribution DocTamper, diffusion inpainting 0.585 TruFor, diffusion inpainting 0.599 0.962 traditional tampering 0.50 0.60 0.70 0.80 0.90 1.00 chance

Sources: When the Forger Is the Judge (humans 0.501, self-judging 0.532, DocTamper 0.585, TruFor 0.599 vs 0.962) and AIForge-Doc (zero-shot judge 0.509, DocTamper 0.563 vs 0.98), both 2026.

Study Measured Result
AIForge-Doc Forged financial and form documents, nine languages DocTamper AUC 0.563 on AI-forged content vs 0.98 in-distribution; a general-purpose model as zero-shot judge, 0.509
When the Forger Is the Judge Diffusion inpainting into documents, with pixel masks TruFor 0.599 (vs 0.962 on traditional tampering), DocTamper 0.585, humans 0.501, and the generating model judging its own forgeries, 0.532
DOCFORGE-BENCH 14 detection methods across 8 datasets, zero-shot Reasonable pixel-AUC but near-zero pixel-F1; the paper's own verdict is that no evaluated method works reliably out of the box
GPT4o-Receipt Humans and models on generated receipts Humans below machine judges overall; the dominant reliable signal was arithmetic, not appearance

Why a whole-image classifier cannot see an edited amount. DOCFORGE-BENCH measured the tampered region at 0.27%-4.17% of pixels. A model that scores the whole page averages a one-percent anomaly into ninety-nine percent of untouched document, which is why the same systems hold a respectable ranking metric and localise almost nothing.

The inputs move too: re-saving at a different quality, printing and scanning, or an unusual JPEG quantisation table each shift the file away from the distribution these detectors were trained on.

So Tamperlens ships no “is this AI?” score and publishes no detection rate. No independent benchmark of document-fraud detectors exists for anyone to substantiate a comparative accuracy claim against, and any percentage offered without a sample size, labelling process and false-positive rate is a hypothesis wearing a number's clothes. What we publish instead is our own measured error rates.

What this cannot tell you

  • Absence of provenance is not evidence of human authorship. Almost nothing writes a manifest into a PDF, and every ordinary re-save strips the ones that exist, 5 of 5 in our measurement. A quiet file is the normal case, not a clean bill of health.
  • A valid manifest does not make the contents true. It binds bytes to a signer, not a salary to a payslip.
  • Valid is not trusted. A self-signed or unknown chain validates arithmetically while proving nothing about who signed. Both reads have to be reported, or the sentence is wrong.
  • A broken manifest means “modified after signing”. Not “tampered”, and not a finding of fraud: re-saving, converting or optimising a file all break a signature exactly the way an intentional edit does.
  • AI-generated is not fraudulent. A supplier typesetting an invoice with a model, an accountant regenerating a statement into a tidier PDF, both produce machine-written documents that are completely honest. What decides a case is whether the claims are true and whether the file changed after it was issued.
  • A confident negative is not available. No structural or statistical check can establish that a document was not model-generated. The deterministic layer can only confirm a positive.
  • None of this replaces the issuer. The bank confirms the payment, the employer confirms the pay stub, the registry confirms the invoice. File analysis is the second opinion, and always a statement about a file, never about a person.

A working order of checks

Every rung above the last one only decides how fast you climb to it.
1 · Ask the deterministic layer Is there a manifest, in the document, or in an image it embeds? Does it validate, and against whose chain? 2 · Read the file's own history Producer and XMP, revision count, what a later revision replaced, what a signature actually covers. 3 · Check the numbers against themselves Balance chains, totals, check digits. The layer that still works on a document generated whole. 4 · A probabilistic score, as a tie-breaker Never the basis of a decision, and only with a false-positive rate measured on your own intake. 5 · Go to the issuer For anything that matters. Everything above it is triage.
  1. Ask the deterministic layer first. Is there a manifest, in the document or in any image it embeds? Does it validate, and against whose chain?
  2. Read the file's own history. Producer and XMP, revision count, what a later revision replaced, what a signature actually covers.
  3. Check the numbers against themselves. Balance chains, totals, check digits. The layer that still works on a document generated whole and never edited.
  4. Treat any probabilistic score as a tie-breaker, never as the basis of a decision, and only with a false-positive rate measured on your own intake.
  5. Go to the issuer for anything that matters.

Automating it

Tamperlens parses the raw bytes itself rather than leaning on a high-level PDF library: load and re-save a file with most of them and the revision history is gone. Eighteen signal families run on every document, each returning a severity, a plain-English detail and machine-readable evidence, all documented with their benign causes in the field guide.

Documents are parsed in memory and never written to disk. The engine is CPU-only, calls no third-party service and is deterministic.

Run the deterministic layer on a document

Open the “Signed, then modified” sample: a precomputed report on a PDF whose signature no longer covers what is underneath it, which is step 1 of the order above with the evidence attached. No account, no upload, no quota. Your own file goes into the same checker: the file is sent over HTTPS, parsed in memory and never written to disk.

Tamperlens reports risk signals, not authenticity verdicts. Signals can have benign causes; combine them with your own decision logic.

Related reading