OCR does not fix a bad scan

Recognition quality is mostly settled before any software runs. By the time a page reaches an OCR engine the decisions that matter have been made — at the glass, in the scanner driver, in whatever compressed the result — and the engine's job is to read the picture it was handed rather than the document that was in front of the scanner.

This is worth being blunt about because the expectation runs the other way. People send a crooked, grey, low-resolution photograph through recognition and are disappointed by the output, as though the software had declined to try. It tried. It read what was there.

What the engine is given, and what it does with it

A scanned page arrives as a grid of pixel values with no notion of a letter in it. Before anything is recognised, the image has to be turned into shapes.

First the page is reduced to marks and background, which means picking a threshold. Then the connected marks are grouped, the layout is worked out into blocks of text and non-text, the blocks are cut into lines by fitting baselines, and the lines into words by looking at gaps. Only then does recognition happen, and in a modern engine it happens over a whole line at a time with a neural network rather than one character after another. A language model scores the result afterwards, which is why a misread word inside a common phrase often gets corrected and the same misread inside a proper noun does not.

Every stage before recognition can fail on its own, silently, and the failure arrives at the end looking like a recognition problem.

The four things that decide the outcome

Skew. Lines are found as roughly horizontal runs of marks. A page a degree or two off is recoverable, and engines deskew. A page far enough off, or one photographed at an angle so the skew varies down the page, gives line-finding a problem it solves badly: two adjacent lines merge into one, or a two-column page has its columns stitched together. Everything downstream is then reading a line that does not exist.

Resolution. The usual target for body text is 300 dots per inch, and the reason is stroke width. A character has to be some tens of pixels tall before its parts are distinguishable, and at 150 dpi the thin strokes of an ordinary serif face are around a pixel wide, which means they survive or vanish depending on where the pixel grid happened to fall. Small print — a footnote, a superscript, the fine text under a seal — needs more, not less. Going much above 400 dpi mostly adds file size and paper texture.

Contrast and evenness. A threshold applied to a page lit unevenly by a window, or by a phone's own shadow, keeps one side and blows out the other. Thin paper gives show-through from the reverse, which arrives as grey text that binarisation turns into speckle sitting among the real characters. And a page already compressed hard as a JPEG carries ringing around every glyph edge; that ringing is high-contrast, so it survives thresholding and attaches itself to the letters as noise. Compressing before recognising is the wrong order for exactly this reason, and it is the order most people use because the size limit is the thing shouting at them.

What is on the page. Print is what these engines are trained on. Handwriting is not supported, and the failure mode is not an error message but confident nonsense — a plausible-looking word where a person wrote something else. Tables, stamps, seals overlapping text, ruled boxes and dot-matrix output are all harder than plain paragraphs, and a signature across a line of print will usually take that line with it.

Mixed scripts cost accuracy

Each language you enable is another set of hypotheses the engine can offer for the same smudge. Two languages on a page that genuinely uses two is right. Two on a page that uses one is a straightforward loss, because you have doubled the ways to be wrong without adding anything to be right about.

The Indic scripts are harder than Latin in a specific way. Devanagari hangs its characters from a headline, and matras sit above and below the base line, so the unit that has to be segmented is not a neat box in a row — a break in the headline, or a speck on it, changes what the cluster reads as. Conjuncts compound this. Urdu in Nastaliq is cursive, right to left, and heavily context-dependent in its letterforms, which makes segmentation into characters close to meaningless as a step.

Numbers deserve their own warning. A dictionary rescues a misread word and there is no dictionary for a registration number, an amount or a date. The classic confusions — zero against capital O, one against lowercase l, five against S, eight against B — all survive into the output looking entirely calm, because a wrong digit is as well-formed as a right one. Anything expensive to get wrong has to be checked against the image by a person.

Why the text goes behind the image

A searchable scan is the original picture with a layer of invisible text underneath it: one text object per recognised word, drawn in the rendering mode that paints nothing, positioned and stretched to the box that word occupies in the picture.

The picture itself is not touched, and that is the design decision worth defending. If recognition replaced the page with typeset text, every error the engine made would become a permanent alteration to the document, and the seal, the stamp, the ink signature and the letterhead would all be gone. The scan is the record. The layer is a machine's reading of the record, and keeping the two separate means the reading can be wrong without the record being damaged.

What follows from that arrangement is not all convenient.

• The file gets larger, not smaller. You added a layer.
• The words are still not editable. Retyping a recognised word changes the hidden layer and leaves the picture saying what it always said, which is a worse outcome than not editing at all.
• What the page shows and what a search finds can disagree. Somebody can search a document, get a hit, and be reading the engine's mistake rather than the page.
• Copying text out gives you the recognised version. Paste it into a letter and you have inherited every error without seeing the original beside it.
• A document that is part typed and part scanned should have its typed pages left alone, or you get two text layers arguing on the same page.

The order that works

Scan straight, in greyscale, at 200 to 300 dpi. Recognise. Check the output. Then compress to whatever limit you are up against, which costs image quality and leaves the text layer alone.

Checking means opening the result and searching for a phrase you can see on the page, then reading the things that would be expensive to get wrong — a number, a date, a surname spelt unusually — against the image rather than against the layer. Creasepoint bundles every language in the package, so nothing downloads the first time you use it, and it reports pages recognised, pages skipped and words found when a run finishes.

Counts like those tell you the run happened. They tell you nothing at all about whether it was right, and no recognition tool's counts ever will, because an engine that could identify which of its readings were wrong would have read them correctly in the first place.