Reading Scanned Pages in Parallel: Design and Test Results

This is a follow-up to Making PDF Conversion Faster. That article ended with a finding: giving our server more CPU made scanned statements convert much faster, even though the code did one page at a time. This one covers the next step, reading several pages at once, why we kept the parsing sequential, how we built it, and what the tests showed.

Summary

• Scanned pages are read with OCR, and each page takes several seconds of CPU. Pages do not depend on each other at that step, so they can be read at the same time.
• Parsing is different. Both of our engines carry state from one page to the next, so parsing stays strictly in page order. Only the OCR runs ahead.
• With four workers, four-page previews of scanned statements ran 2.0 to 3.4 times faster than with one, and the rows were identical.
• One document did not speed up. Its time appears to go mostly to something other than OCR, probably AI header detection, but I did not profile it.
• These results are from a development machine. We had not yet timed this on the live service when this was written.

Which parts can run at the same time

Converting one scanned page involves two kinds of work. The first is OCR: render the page, straighten it if it was scanned sideways, and run Tesseract to get every word with its position. That work depends only on the page's own pixels. The second is parsing: turning those words into rows. That work reads and changes state that later pages rely on.

The bank statement engine carries all of this from page to page:

• Whether the document is a credit card statement, which changes how amounts are signed for every later page.
• Whether dates are written day-first, once a page has shown it.
• The section heading currently in effect (for example deposits versus withdrawals) and the positions of the deposit and withdrawal columns.
• The statement period and year, which fill in dates that print only a month and day.
• The last few lines, used as context when the AI has to read a hard line.
• A transaction whose description wraps from the bottom of one page onto the next.

The column engine reads its column headers from one page and applies those column positions to every later page. It also skips repeated header lines and keeps a running count against the transaction limit.

The AI page check is a third kind of work. In the column engine each check needs only the headers, the page's rows and its image. In the bank engine the cleanup after the check uses the statement year and month as they stood at the end of that page, which later pages can change. Running those checks afterward, in parallel, would need that state recorded per page first. We left it out of this round.

So the split is: OCR ahead in parallel, parse in order. The parsing code did not change at all.

Why a sliding window

We considered three ways to schedule the OCR.

We built the third one.

How it works

1. Start of the page loop. As the parser begins page N, a small pool of worker threads is asked to read pages N through N plus the number of workers.
2. The parser asks for a page the same way it always did. If the page is already read, it gets the result immediately. If a worker is reading it right now, the parser waits for that worker instead of starting a second OCR of the same page. If nothing has started on it, the parser reads it itself.
3. Each page is read once. Results are kept in a small in-memory cache keyed by the file, its size and its modified time, which also serves the word-position and plain-text needs of the column engine.
4. Bank engine. It prefetches only pages whose text layer is poor, which are the pages it would OCR anyway. A PDF with a good text layer is never OCR'd by the prefetcher.
5. Column engine. It reads word positions for every page, so it prefetches every page.
6. Previews prefetch exactly the pages the preview reads, so a two-page preview never starts work on a third page.
7. Leaving the loop. When the loop ends, for any reason, pages that were queued but not started are cancelled, the few already running are allowed to finish, and the cache is cleared when the job ends. No OCR outlives the job.

How many workers

An OCR_WORKERS setting, from 1 to 8, decides. If it is not set, the code reads the container's CPU limit and uses that, capped at 4. If it cannot tell, it uses 1, which turns the prefetch off and gives the same behavior as before. Each Tesseract run is limited to one thread so that the page-level workers supply the parallelism, not several threads inside each run.

A lock came back

In the previous round we removed a lock around the OCR cache, because the server handled one request at a time with one thread. With worker threads sharing the cache, the lock is needed again, so it is back. It is held only for short dictionary operations and never during rendering or OCR.

How we tested it

We ran the real engines on eight documents, reading the first four pages of each: three scanned bank statements (a credit card, a checking account and a bank statement) through the bank engine, one accounts payable aging report and two scanned card statements through the column engine, and two PDFs with a good text layer. The AI page check was turned off, so the comparison covers only the deterministic code.

Each document ran at 1 worker and at 4 workers, each in a separate process, and we repeated the whole thing in a second pass. The machine has 24 logical processors.

Timing

Times are seconds to read and parse four pages. The first pass at one worker was far slower than the second for the same code. I could not confirm why. The computer may have gone to sleep during it. The second pass is the one to read.

Across the five documents that sped up, the range was 2.0 to 3.4 times. The two text-layer PDFs show that the prefetcher costs nothing when no page needs OCR.

The scanned card statement 2 did not improve, and it was 84 seconds at four workers in the second pass. Most of that document's time seems to be something other than OCR, probably the AI calls that find the column headers and can retry. I did not profile it, so I cannot say for sure. Either way, the OCR saving is lost in call-to-call variation. I would not read that row as a slowdown caused by the prefetcher, because the first pass showed the opposite (77.6 seconds to 54.7), but I also do not have a measurement that shows a gain for it.

Did the output change?

For the three bank-engine scans, the accounts payable report and the two text-layer PDFs, the rows were identical across all four runs. For the two scanned card statements run through the column engine, the row counts varied between runs:

The same setting gave different counts against itself. Card statement 2 produced 0 rows at one worker in the first pass and 178 in the second. That most likely points to the AI header detection, which is not deterministic, and not to the prefetcher. I did not isolate it. It is also a flag worth following: a document returning 0 rows on one run and 178 on another is a reliability problem in its own right, and it is separate from this change.

To check the OCR itself, we ran our regression suite with four workers. It compares the OCR page text, the document type, the engine chosen and the rows for eight documents against a capture from before the prefetcher existed. All eight matched. The 27 unit tests passed.

Failure and early-stop tests

The second test matters most. A worker that fails does not break the job. The parser still asks for the page, finds nothing cached, and reads it the way it always did. The prefetcher can only make a job faster or leave it as it was.

Memory

In the earlier measurement, OCR on one page peaked at about 400 MB including the Tesseract and poppler processes. Four pages in flight, plus the page the parser is on, should add roughly 1.3 to 1.6 GB at peak. The server has 4 GiB. I estimated this from the single-page measurement and did not measure peak memory with the prefetcher running.

What we have not shown

• Live numbers. The results above are from a 24-processor development machine. The live service has 4 CPUs, so its gain may differ from these. I will add its numbers once they exist.
• Whole-job time. The AI page check was off, so these are OCR-plus-parsing times, not what a user waits for end to end.
• Why some timings varied. Single runs on a laptop that may sleep are noisy. The speedups are consistent in direction and size across five documents, but they are not precise to the second.
• Long documents. Every test was four pages. The window is meant to keep memory bounded on long files, and I have not run one through it.

To see how your own scans convert, try the converter with 50 free transactions. For supported formats, see supported banks.