Reference: Detect an edited PDF
I built 1,006 forged PDFs and ran my own engine over them, because nobody hands you that corpus. False positives were the easy half: point the engine at 1,728 published documents nobody curated, then count how often it complains. Recall needs documents known to be tampered, and no public set labels them that way. The academic forgery corpora are pixel-level image forensics. The PDF corpora hold malformed and malicious files, which is a different thing: a malicious PDF attacks the reader, a forged one lies to it.
So I built the positives myself. I started from 160 genuine documents,
published by their own issuers, drawn from the two uncurated populations. Every
one scored low with a single revision before I touched it, so no
false positive is hiding inside the result.
Then eight forgery operations on each. Four are direct edits: a box painted over a value, a black block, a retyped line, a spliced image. Three add laundering on top: a metadata strip, a qpdf rewrite, a Ghostscript re-distill. Those last two are command-line tools that rewrite a PDF from end to end. The eighth composites the edit through a Form XObject, which is what an online PDF editor writes. 1,006 positives in total.
Proving they are forgeries, not broken files
A script that corrupts files and calls them forgeries measures nothing. Three checks stand between the generator and the numbers.
qpdf --check ran over the whole set: 944 clean, 62 warnings, 0
broken. Every warning traces back to one source that already had them. I also
rendered the pages and looked at them.
The covering box aims at a text run I know is still visible, not at a guessed band. A box over white space hides nothing. It would have measured the script's aim rather than the engine.
And I applied every edit to the publisher's real bytes, as a genuine
incremental update: a classic cross-reference table plus a /Prev
pointer back to the previous one. Editing a normalised copy would have
measured qpdf instead.
The first run, and the row that ruined the day
| operation | n | caught |
|---|---|---|
| incremental cover box / black box / retype | 426 | 100.0% |
| spliced image | 112 | 100.0% |
| edit, then metadata strip | 110 | 100.0% |
| edit, then qpdf rewrite | 97 | 71.1% |
| edit, then Ghostscript re-distill | 110 | 66.4% |
overlay-retype, a Form XObject composite | 151 | 0.7% |
The top of the table is the reassuring half. Any edit saved as an incremental update, which is what Acrobat writes by default, is caught every time. Stripping the metadata afterwards does not help the forger, because the strip adds a revision rather than removing one. Whole-file laundering degrades detection without defeating it. A qpdf rewrite drops it to 71.1%, a Ghostscript re-distill to 66.4%.
The last row is the reason to publish the table. overlay-retype
is the same visible forgery, produced by a tool that composites rather than
appends. 151 files, one caught.
The diagnosis was in the data, not in a guess. All 151 carry one revision and keep the original producer string. In the sample, every file had Form XObjects while the covered text runs detected were zero of 40. The white box and the fake number live inside a Form XObject. The content walker, the part of the engine that reads a page's drawing instructions, never stepped inside one.
The report had disclosed exactly that, on every affected file, in a field
named hasFormXObjects. Disclosure was the honest choice. It did
not make the miss any cheaper: every online-editor forgery was invisible.
Entering the form: 0.7% to 89.3%, and what it cost
Engine 1.18.0 makes the walker step inside Form XObjects. Three things had to be shared across that step rather than kept per stream. The first is a correctness property: paint order. The box and the text it hides sit in two different streams. Only a counter that spans both can say which came first, and that is the entire question the redaction family asks. The operator ceiling and the retention caps also became global, because a per-stream cap bounds nothing when the attacker chooses how many streams there are.
Re-running the 160 genuine sources showed the cost. One of them turned
elevated. The culprits were an e-signature platform's seals and
a letterhead logo: transparent images whose bounding boxes crossed the
heading beneath them.
Every one declared an alpha mask, the transparency layer that says which pixels actually paint. So the rule followed: a masked image is no longer a cover. Counting one would mean reporting every line of text that passes near a logo.
What survived on that document was a single opaque banner drawn over a
heading that really is underneath it. It reports medium, and it
is true. After the clip-path work in 1.19.0, the confound count on the recall
run is 0 of 1,006.
| operation | caught |
|---|---|
| incremental edit, spliced image, edit then strip | 100% |
| composited through a Form XObject | 89.4% |
| edit, then qpdf rewrite | 71.1% |
| edit, then Ghostscript re-distill | 65.5% |
Those are the rows on the evidence page, stamped engine 1.23.0 and identical to the decimal across the 1.19.0 and 1.22.0 re-runs. The residue of the form walk is disclosed rather than fixed. A form nested past depth 8, repeated past 2,000 invocations, or one that will not decode is still unread. The report says so on the affected file.
The number you will not find here
The eight operations average to 78.6%. That figure appears nowhere on this site. Averaging them assumes a forger picks uniformly among the eight operations. Nobody measured that assumption, and it is certainly false. It is also exactly the kind of number that turns into a marketing figure with no denominator.
Here is the narrower sentence that replaces "high precision, unknown recall". Naive forgery is caught every time. A Form XObject composite is caught roughly nine times in ten. Whole-file laundering drags detection down to about two thirds.
These are the forgeries I thought of. Eight operations applied by a script
are not an adversary. And a fully regenerated document, laid out fresh with
the fraud in it from the first byte, contains no edit to find. It scores
low. That class is measured separately on the evidence page,
as a null result, because publishing it is cheaper than letting a customer
discover it.
Reproduce the shape of it
You do not need my corpus to see the top row of the table. Take any PDF you own and change one word in an editor that saves incrementally. Then inspect the result. The appended revision, the kind of object it wrote and the band it earns are all in the report.
Check a PDF: free, no account, nothing stored. The before-and-after pair takes about a minute to make and is the most honest demo this product has.