Research

What actually leaks when a PDF redaction fails

We checked 5,987 public U.S. federal court filings that had been marked as redacted. In 1,879 of them, someone had covered text with a black box. In 22, the text was still there underneath — in the file, recoverable by selecting it and pasting it somewhere else.

Here is what was under six of them.

The other 16 ran the same way. Among them: the price a contractor bid against the government's own estimate, a customer list, a product's annual revenue, a sealed complaint, patent specifications, a sealed expert opinion, trading records belonging to a sanctioned entity, a police officer's fitness-for-duty evaluation, and the destination a defendant asked the court for permission to travel to.

None of it was trivial, and that is the whole problem. People redact the part they had a reason to hide, so when redaction fails, what escapes is the part that mattered.

Why it happens

A PDF holds two things: the page you see, and the text that page was built from. Drawing a black rectangle changes the first and leaves the second untouched. The document looks redacted on screen and in print, and the words are still sitting in the file, one copy-paste away.

Nobody does this on purpose. It happens because the tool people reach for — a highlighter, a shape, a marker — does exactly what it looks like it does, and nothing more.

How we checked

We took public filings from the CourtListener archive whose docket entry described them as a redacted version, then tested every one: a word counts as leaked only if it is still in the PDF's text layer and invisible on the rendered page because a solid dark box covers it. Six sample documents with known answers — including the table borders, dark letterheads and photographs that fooled our first attempt — are run before every pass. All 22 findings were then confirmed by eye, page by page — 121 leaking pages in total. The filings came from 87 different courts.

Two things worth knowing about the numbers. Our sample came from filings whose own docket text announced a redacted version — filers who knew they were redacting and meant to do it properly — so it cannot tell you how often this happens across the courts generally. And the test only sees near-black boxes over real text: grey boxes, coloured boxes and redactions applied as flattened images all pass it silently. The true count is higher than 22.

The detector is here, as we ran it: detector.py, make_controls.py, analyze.py, collect.mjs and the aggregate results. Run it on your own sample. If you get a different answer, tell us and we will publish it.

Redact a PDF so the text is actually gone

PDFRedax rebuilds redacted pages without the text underneath, so there is nothing left to select or copy. It runs entirely in your browser — the file never leaves your device — and the first page is free, with no account.

Questions

Does a black box over text redact a PDF?

No. A PDF keeps the picture you see and the text separately. A black rectangle covers the picture; the text stays in the file, and anyone can select it, copy it, or pull it out with one line of code. Only removing the text removes the text.

How do I check whether a redacted PDF is safe?

Open it, drag across the area under a black box, copy, and paste it somewhere else. If anything appears, the redaction failed. It takes about ten seconds and works in any PDF reader.

How do I redact a PDF properly?

Use something that deletes the underlying content instead of covering it. PDFRedax rebuilds redacted pages without the hidden text, in your browser, so the file never leaves your device and nothing can be recovered from the copy you share.