Three ideas from pdfid, measured on 5,168 documents first

Didier Stevens's keyword scanner has been triaging hostile PDFs since 2009. I evaluated three of its ideas for this engine, and the house rule got to decide which ones shipped.

Reference: Check a PDF for malware signals

pdfid.py is one of the oldest tools in PDF triage: a scanner that counts the keywords a hostile document tends to carry. Most of what it counts, this engine already reads. Three of its ideas were things mine did not do, so on 24 August I ran all three through the measurement the house rule requires before any signal is allowed a severity.

Two shipped. One did not, and the number that killed it is the best part.

The rule that decides severities

No signal in this engine ships with a severity before a false-positive measurement on a benign corpus. The rule exists because of a specific embarrassment: /OpenAction, the auto-run flag every PDF security checklist tells you to fear, fired on 189 of 213 documents in a random .gov sample when we finally measured it. An auto-open action is how a table of contents works. It has reported at info since engine 1.17.0, and every candidate since inherits that lesson: the number first, the severity after.

The corpus for this run is the local artefact corpus the other phase-0 measurements use: 5,168 PDFs from Brazilian banks and issuers, public procurement contracts, government sites and document templates. 0 unreadable, 28 encrypted, 29 truncated by a parse budget. The run is one script, checked in:

The whole measurement

npx tsx scripts/corpus-didier-phase0.mjs --out /tmp/didier.jsonl

A name that hides itself: /J#61vaScript

A PDF name may spell any byte as #XX. Some bytes need it: a space or a slash inside a name has to be written #20 or #2F, and legitimate producers do exactly that. No byte that is a letter or a digit ever needs it. There is exactly one reason to write /J#61vaScript instead of /JavaScript: a string scanner reading raw bytes never sees the keyword.

The same dictionary, as a string scanner sees it

/OpenAction << /S /J#61vaScript /JS (app.alert…) >>
                    └── decodes to /JavaScript; grep never matches

Tamperlens parses names byte-by-byte, so it decoded these correctly all along. The escape was never an evasion that worked here. But the parser was throwing away the interesting fact: that the name had been hidden at all. Since 1.33.0 it keeps two records. One says whether any name in the file escaped an alphanumeric byte. The other says whether a name from pdfid's dangerous set (JS, JavaScript, AA, OpenAction, Launch, JBIG2Decode, RichMedia, EmbeddedFile) matched only after such an escape.

The measurement is the reason this one leads its family: not one of 5,168 benign documents hex-escaped a single alphanumeric byte of any name. Zero on the broad count, zero on the sensitive count. The practice does not occur in legitimate output at all, so a sensitive name that appears only behind an escape is a near-certain marker of deliberate concealment, and it reports at medium, above the executable content itself.

It is also self-incriminating by construction. Because the decode still sets the underlying presence flag, the report shows both facts at once: the file carries the action, and the file tried to hide it.

The severity of /AA is decided by its margin, not its rate

/AA is /OpenAction's per-event sibling: an action fired when a page opens or a form field gains focus, loses it, or formats itself. The engine now detects it. What severity it deserved is entirely a question about numbers, and the numbers say something more specific than "it is rare":

measured on 5,168 benign PDFscountrate
carry /AA at all260.50%
carry /AA with no JavaScript flag90.17%
carry /JavaScript (context)340.66%
carry /OpenAction (context)2605.03%

The row that matters is the second one. When an /AA sub-action holds a script, the parser's inline /S /JavaScript walk already fires the JavaScript flag at medium, so the only documents /AA detection adds anything on are the ones carrying it with no script: nine documents, all per-event navigation and formatting. That marginal case is the same class as a bare /OpenAction, so it reports the same way: info. A severity assigned on the presence rate alone would have double-counted everything the engine already says.

The one that was killed by its own population

The third candidate was /XFA: a dynamic-forms surface whose XML dataset can diverge from the static render, meaning a document can show one value and carry another. That is classic filled-form tampering territory, and building divergence detection for it was genuinely tempting.

The measurement: an upper-bound string scan, the same technique pdfid itself uses, matching a /XFA token anywhere in the bytes. It found zero of 5,168. Since the scan over-matches by design, the true dictionary-scoped population is at most zero. On top of an absent population, XFA is deprecated in PDF 2.0 and Acrobat is removing it. An XML-divergence parser for a surface that is absent today and shrinking tomorrow is effort spent on decay, so no flag ships and nothing in the engine reads XFA. The string scan stays in the repository as the cheap re-check, in case a future corpus disagrees.

What deliberately did not move

None of this is a new signal family. All three ideas land in active-content, whose mandate (what the file can do; never executed, only reported) already fits them, so the published count of eighteen families is unchanged and the family still folds to one signal per report. The severity ladder gained one rung at the top, concealment, and /AA joined /OpenAction on the bottom one.

The evasion bench re-ran because the engine version moved, which is the bench's own rule rather than optimism. None of its 92 fixtures carries a hex-escaped name or an /AA, so the matrix came back changed in exactly one place: the version stamp.

Caveat

These rates are properties of this corpus, which is weighted toward Brazilian banking, procurement and government output, not of PDFs at large. A benign rate of zero is still a statement about 5,168 specific files on one day. And a concealment marker at medium is a signal about bytes, not a verdict about the person who sent them.

Read a file's own account of what it can do

The active-content evidence block in every report lists the presence flags this post is about, and since 1.33.0 it names an obfuscated keyword when one exists. If you have a PDF you did not expect to carry JavaScript, that block answers in one look.

Check a PDF: free, no account, and nothing is stored. The file is parsed in memory and the report comes back with the evidence under every finding, including the flags that stayed at info and why.