The PDF was going through untouchedThe PDF was going through untouched

hushgate sits between an application and a cloud LLM API. It finds the personal data in an outgoing request, swaps it for stable placeholders, forwards the sanitised version, and puts the real values back into the response. Your code still sees anna.schmidt@nordlicht.example; OpenAI only ever sees [EMAIL_1].

That works because the redaction layer walks the request body and hands the string leaves that carry prose to the detectors. A model name is left alone, a tool identifier is left alone, and the message text is not. It is a deliberate, narrow thing, and it is the reason a placeholder never lands in the middle of a JSON key.

It also has an obvious hole in it, which I did not see until I went looking. A base64 PDF inside a content part is not a string leaf that carries prose. It is a string leaf that carries three megabytes of base64. So it was skipped — and forwarded, byte for byte, to the provider.

The one document type a business actually attaches was the one thing going through unredacted.

The comment in the source even said so, in the file that decides which paths get redacted: image parts are deliberately absent, because a data URL is megabytes of base64 that no detector can validate anything in. That reasoning is correct for an image. It quietly extended to every document anyone attached.

The shape of the fix

The fix that took the least new surface was not to teach the redactor about documents. It was to make sure there are no documents left by the time the redactor runs.

readBody → parseJsonObject → rewriteAttachments → redactJson → upstream
                            ^^^^^^^^^^^^^^^^^^ new

A pre-pass walks the body, finds anything attachment-shaped, turns it into text, and puts a plain text content part where the file was. By the time redaction runs, the invoice is prose like any other prose. Every detector, every policy, the placeholder table, the audit trail and the streaming re-hydration all apply to it with no change at all. The detector layer got zero new lines.

What the provider receives is exactly what you would write by hand if you had read the file yourself:

Worum geht es in dieser Rechnung?

--- attachment: Rechnung [NAME_1].pdf (application/pdf, 1 page) ---
Rechnung Nr. 2026-0815
Kundin: [NAME_1]
E-Mail: [EMAIL_1]
Telefon: [PHONE_1]
IBAN: [IBAN_1]
--- end of attachment ---

The filename is pseudonymised along with the contents, and that is not incidental. Kuendigung_Anna_Schmidt.pdf names a person and discloses an employment event before anyone opens it. It belongs in the prompt, where the detectors will catch it — and nowhere near the audit trail, where nothing pseudonymises anything.

Not parsing the PDF myself

hushgate has zero runtime dependencies, and that is not decoration. It is most of the argument: no third-party code touches your documents, nothing phones home, and the whole test suite passes with the cable pulled out. So the obvious move was to write the PDF extractor too.

I wrote it. Then I measured it, and threw it away.

On ordinary documents — a Google Docs export, a letter from a telecoms provider — it produced output that read fluently, was about the right length, and had the recipient's name, street, postcode and customer number simply missing from it. Not garbled. Absent. A parser that returns plausible prose with the personal data quietly removed is the worst possible component to put inside a tool whose entire claim is that personal data does not get through.

Bundling someone else's JS engine was not better. The smallest credible option ships 1.67 MB of minified Apache-2.0 code under a licence file that does not mention it — a blob you cannot read, carrying an attribution defect you would inherit. In a thing people point auditors at, that is a strange asset to acquire.

So PDF extraction goes to a separate process the operator configures — pdftotext, which has been doing this for twenty years. The document is piped to its standard input, nothing derived from the request ever reaches its arguments, no temporary file is written, and the child gets a minimal environment rather than the proxy's, which holds your provider API key. The npm package still has zero dependencies. The Docker image installs poppler, so the deployment people actually use reads PDFs out of the box.

The argv rule sounds fussy until you try it. A file named -layout, handed to pdftotext as an argument, is read as a flag: the tool exits zero and produces nothing. A file named -l 1 truncates a four-hundred-page document to one page, also exiting zero. Attacker-chosen argv is a silent-failure machine, so the request only ever reaches the child on stdin.

The failure that is not an error

Here is the thing I would take to any other project. When you shell out to a converter, you check the exit code. That is not enough, and the case where it is not enough is the case you care about most.

pdftotext on a scanned page — a fax, a photographed contract, the sort of thing a German Mittelstand company has by the filing cabinet — exits 0 and prints a single form feed. That is indistinguishable, to anything checking status, from a document that genuinely contains no text. If your fallback on empty output is to forward the original, a scan full of patient names has just gone to a provider with a clean exit code and a log line saying everything worked.

So extraction is not believed on the strength of exiting zero. It has to clear floors: enough characters overall, enough per page, few enough replacement characters, few enough control characters. Anything that fails is unreadable, and an unreadable document is refused with a 422 rather than forwarded. That default is the whole posture — a document hushgate cannot read is a document it cannot pseudonymise, and the safe answer is to say so loudly.

Dropped text is not the same as shredded text

There are two ways an extractor mangles a document, and they are not equally bad.

• It drops a paragraph. The text hushgate never saw is text it never forwards either, so nothing leaks — the model just gets less to work with.
• It splits the words. That text is forwarded, matches no detector, and reaches the provider in clear. That one is a leak.

I assumed the second was exotic. It is not. Ordinary PDF kerning — a TJ array with an offset every few glyphs, which is just typesetting — comes out of a real pdftotext like this:

Sac hbe arb eit eri n: Ann a S chm idt
E-M ail : a nna .sc hmi dt@ nor dli cht .ex amp le
IBA N: DE8 937 040 044 053 201 300 0

Every identifier in that is intact to a human and invisible to a detector. It has no short words in it, so a fragmentation ratio reads it as fluent prose. Length checks pass. Replacement-character checks pass. And the name, the whole address and half the IBAN leave the machine, with the audit trail recording the document as read and pseudonymised.

Measuring the right thing

My first attempt was statistical: count how many words are one or two characters long, over a sliding window so a shredded address inside a clean page could not hide in the average. It caught the shredding. It also refused a business letter containing a table of country codes, a bibliography, and a price list with ME and USt columns — because a window small enough to catch a mangled address block sits entirely inside a table of short codes. For a tool aimed at invoices, refusing invoices is not a rounding error.

The statistic was the wrong instrument. It measures how the text looks, when the question is whether anything is hidden in it. So the last check asks that directly: run the detectors over a window of the text, then over the same window with its spacing closed up, and see whether closing the gaps reveals something that was not visible before.

A shredded address closes up into an address, and is refused. A column of country codes closes up into DEATCHFRIT, which is not an identifier and reveals nothing, so the table is left alone. Same instrument, both cases, no threshold to tune.

Getting there took two adversarial review passes that between them found twenty-eight real defects in my own work, every one reproduced against the running proxy before I touched it. Two were leaks where hushgate decoded a document and wrote the plaintext into a corner of the request that redaction never visits — the audit record said extracted, findings none, and the IBAN went out in clear. That class is closed structurally now: extracted text is only ever placed where the route's own redaction rules will visit it, and a shape that does not qualify fails towards refusing rather than forwarding.

What it still will not do

There is no OCR, so a photograph or a scan has no text to read. It is refused rather than forwarded, but refusing is all it can do. The spacing check can only notice the loss of an identifier the detectors would have found in the first place — a street address is not one of them. And it closes up spacing, not line breaks: an extractor that broke an address across a line every few characters would defeat it. I left line breaks alone deliberately, because the alternative refuses invoices with a narrow column of figures, and that is the document this gets pointed at most.

I would rather write those three paragraphs than imply the guarantee is total. The reason anyone runs a thing like this is to be able to answer a question honestly when their data protection officer asks it, and a control whose limits are not written down is not a control you can point at.