What a portal means when it asks for 300 KB

When an examination portal asks for a certificate between 50 KB and 300 KB, the second number is the one everybody reads and the first is the one that fails them. A file under the floor is refused as firmly as a file over the ceiling, usually with the same unhelpful message, and most compression tools model the requirement as an upper bound alone and will happily hand you a 38 KB document.

The floor is not decoration. Somebody on the other side of that form has to read the scan, and a scan small enough to slip under 50 KB is a scan where the registration number has stopped being legible. The band is a crude proxy for is this actually a readable copy of the document, and a crude proxy is what a web form can enforce.

Why a scan weighs what it weighs

Take an A4 page at 300 dots per inch, which is the usual target for a document that has to be read rather than glanced at. A4 is 8.27 by 11.69 inches, so the page comes out at roughly 2,480 by 3,508 pixels. That is about 8.7 million of them.

Stored as full colour, at three bytes each, the raw page is around 26 MB. In greyscale it is a third of that. As pure black and white, one bit per pixel, it is about 1 MB before any compression at all. Those numbers are the material a compressor is given, and they explain the whole problem: the file is large because the page is a photograph, and a photograph of a page is an enormous amount of data describing something whose actual information content is a few hundred words.

A page of real text is the opposite. The glyph-drawing instructions for a page are a few kilobytes, and the font travels once for the whole document rather than once per page — often as a subset holding only the characters the author actually used. This is why a forty-page bank statement generated as text can be smaller than a single scanned page, and why merging a text document with one scan produces a file whose size is entirely decided by the scan.

What compression discards

Two different things go on under the word compress, and only one of them is what you are asking for when a portal has set a limit.

The lossless half rewrites the file's structure: streams get deflated, objects get packed together, redundancy goes. Nothing about the page changes. On a text document this recovers a few percent, and on a scan almost nothing, because the image was already compressed when it was made.

The lossy half re-encodes the images, and there are three levers.

• Quantisation. JPEG works on eight-by-eight blocks of pixels, converts each to frequency coefficients, and divides those by a table before rounding. Lower quality means a coarser table and more coefficients rounded to zero, and the high-frequency ones go first. Sharp edges are exactly what high frequency means, and a page of printed text is nothing but sharp edges. That is why heavy compression on a scan shows up as haze around the letters and thin strokes fading out, while the same setting on a photograph of a landscape looks fine.
• Downsampling. Fewer pixels per inch. Going from 300 to 150 halves each dimension and so quarters the pixel count, which is the biggest single saving available and the one that most directly costs you small print — a footnote, a superscript, the matras above a line of Devanagari.
• Colour. Dropping to greyscale removes two thirds of the data on a page that was only ever black ink on white paper. Dropping to bilevel removes more again.

Bilevel deserves a warning. One family of compressors for black-and-white scans works by finding repeated shapes on the page, storing one copy of each and referring back to it. Run in a lossy mode, that matching can decide two characters are the same shape when they are not, and substitute one bitmap for the other. The result is a clean, sharp, entirely convincing page with a wrong digit in it. There is no visual artefact to warn you, which makes it worse than the blurring everybody worries about.

All of this is one-way. Compressing twice does not compound the saving in any useful sense but does compound the loss, because the second pass quantises coefficients that were already quantised. If a file has been round-tripped through a compressor twice, the second pass took quality out and gave very little back.

What compression cannot touch

A file that is mostly text and vector graphics has nothing left to re-encode. A word processor export, an e-filing acknowledgement, a bank statement pulled from a website: the pages are instructions and font data, and a compressor can shave the structure and then it is finished. Expect a few percent. Anything promising more than that on such a file is either about to strip the embedded fonts, which changes how the page renders on a machine that does not have them, or is about to render the pages to images, which makes the file larger and destroys the text at the same time.

When the file is too big and the content is text, the answer is fewer pages rather than harder settings. Sending the three pages that were asked for instead of the whole forty-page statement solves the size problem and the reviewer's problem at once.

Get it right at the scanner

Most files that fight a size cap were made badly at the start. A phone photograph of a page, in full colour, at whatever resolution the camera defaults to, is perhaps 4 MB of mostly noise, and squeezing that to 200 KB throws away the wrong things because the useful signal is a small fraction of what is there.

Scanning the same page in greyscale at 200 or 300 dpi, straight, with the lid closed, produces something that lands near the band on its own and survives compression when it needs to. Recognition wants the same conditions, so if the document also has to be searchable, run OCR on the good scan first and compress afterwards. The other order recognises a degraded image and gets a degraded result.

And read the notice rather than a summary of it. Portals change limits between cycles, and the format requirement is separate from the size requirement — a PDF at exactly the right size is still refused by a form that wanted a JPEG. Some caps apply to each attachment, some to the whole submission, and the two are easy to conflate at eleven at night on a deadline.

Aiming at a band

A compressor that only knows about a ceiling has one job and can always do it, by making the file worse until the number is small enough. A band is harder and more honest, because it has three outcomes rather than one. Creasepoint's compression reports which of the three happened: the file is already inside the band and was left alone, because re-compressing it would cost quality for nothing; the file is under the floor, and compression is the wrong direction entirely; or even the hardest acceptable setting stays over the ceiling, in which case it names the smallest size the pages survive at instead of handing back something unreadable at the number you asked for.

Whatever produces the file, open the output and look at it before you attach it. Check the seal, the signature, the photograph and any handwritten number, because those are the four things a portal's clerk will reject the submission over, and they are the four things a lossy setting degrades first.