A 3-billion-parameter optical character recognition model has achieved 97.6% accuracy reading historical books, according to a new leaderboard released by Hugging Face and EleutherAI that tested 14 open-weight models against expert transcriptions. The FineBooks benchmark pits these models against 2,165 hand-verified pages pulled from the Biodiversity Heritage Library, a collection holding more than 300,000 digitized natural-history volumes spanning 64 million pages. The results show that several lightweight models can now match or approach the performance of much larger systems while running at a fraction of the cost.
The top performer, dots.mocr at 3 billion parameters, hit 97.6% reading accuracy on the test set. Two smaller models followed closely: OvisOCR2, with just 0.9 billion parameters, reached 96.9% accuracy, and PaddleOCR-VL-1.6 at 1 billion parameters scored 96.1%—both processing a thousand pages for less than fifty cents. A much larger system, olmOCR-2 with 8.3 billion parameters, scored 95.7%. The most striking pattern is that multiple models under 2 billion parameters effectively matched bigger ones on this task while costing far less to operate. All 2,165 ground-truth pages came from six book volumes in English, French, German, and Latin, all set in Antiqua-family typefaces.
The report divides the "good enough" question into three distinct use cases rather than offering a single judgment. For LLM training corpora, leading models outperform legacy library OCR pipelines and re-processing entire collections is now economically viable, with the authors citing a Talkie experiment showing models trained on OCR-derived text learned at 30% of the efficiency of those trained on human transcriptions. For libraries weighing replacement of existing pipelines, the verdict is uncertain, because the new models output Markdown or plain text instead of the ALTO XML that library systems depend on for word-level coordinates. For scholarly transcription requiring faithful character preservation, the answer is no: the models quietly modernize archaic letterforms like the long s and various ligatures, acceptable for language models but unsuitable for work demanding exact reproduction.
The benchmark's scope is deliberately narrow, the authors acknowledge. Fraktur typefaces, non-Latin scripts, handwriting, multi-column layouts, and materials like newspapers, magazines, and archival documents all fall outside the test set, so top-line accuracy may not transfer to those formats. Next, the team plans to run the winning models across roughly 200,000 public-domain items in the Biodiversity Heritage Library and release the resulting text, datasets, page classifiers, and processing pipelines openly. If accuracy holds beyond the small benchmark set, a substantial portion of public-domain scientific literature will shift from low-quality legacy OCR to usable training and search data for anyone building open models. The transparency around where these models succeed and where they fail sets a practical floor for what's deployable today versus what still requires human review. Organizations training large language models now face a choice between speed and fidelity that wasn't cleanly quantified before, and the economics of smaller models may reshape which institutions can afford to digitize their archives at scale.