Recognizers for the Livro de Registro de Leis nº 1 (São Carlos, 1892–1894)

Kraken line recognizers fine-tuned from PP-OCRv6 medium (Kiessling, Zenodo 10.5281/zenodo.21788410) on 19th-century Brazilian Portuguese handwriting. They are used by the pipeline at DeguShi/OCR-pipeline-historical-data.

file version SHA-256
st_c1m.safetensors v6 (final) 8b1488d66916357ab9ad1770e5e53777573295f5d2bbc987fb25c717a75ad15a
st_g3m.safetensors v5 d41ac347a293d5d7a3588a7a0b771aa7a9332f3a540b13f7ded18f11fd1eee4d

Training

  • Data: line crops from 78 pages of the same register. The labels are diplomatic transcriptions produced with Gemini 4 Argon and aligned to the segmented lines. They were not reviewed by a person.
  • Never seen in training: the 12 pages with an official transcription and the 10 test pages.
  • Setup (st_c1m): 20 epochs, learning rate 1e-4, augmentation, seed 42, one NVIDIA L4.

Results (full pipeline, v6)

The numbers below come from the whole pipeline, which adds a period dictionary and a trigram context model; they are not the recognizer alone. On 3 held-out pages, against the official transcription: WER 18.74%, CER 4.43%. On 13 pages outside training and selection, against the diplomatic reference: WER 4.10%, CER 1.68%.

Use

git clone https://github.com/DeguShi/OCR-pipeline-historical-data.git && cd OCR-pipeline-historical-data
pip install -r requirements.txt && python setup_pipeline.py --skip-install   # downloads and checks st_c1m
python -m ocr_pipeline.batch --config configs/final_v6.json --images my_pages/ --out outputs/batch

Limitations

The recognizers were trained on a single hand and a single book. They also learned the notation of their labels, for example 1.º for ordinals. The source images belong to the Fundação Pró-Memória de São Carlos.

Post-correction baseline: ByT5-small (postcorr_byt5_small/)

Baseline C of the post-correction comparison (not used by the pipeline). google/byt5-small trained to correct st_c1m's raw greedy line text, on 80k synthetic pairs: clean lines from the 78 training-page transcriptions and ALESP laws 1889–1910, corrupted with character substitution/deletion/insertion statistics estimated on the 9 tuning pages. No held-out or test page was used. 1 epoch, batch 16 × 2, lr 5e-4, max 128 bytes, seed 42, one T4 (3,395 s), transformers 4.51.3 (env.txt holds the full pip freeze).

model.safetensors SHA-256: cf52bdeeddc4cdc166afb745f3c462d1fd4ba903fa269af66d380563c2cbf078.

This is a retrain with the same code, data and seed as the run reported in the paper (whose weights were lost); GPU training is not bit-exact. WER of this checkpoint (beam 4): held-out 21.89%, test 6.81% (paper run: 21.26%, 6.86%; v6: 18.74%, 3.49%). Code: tools/baseline_byt5.py; records: records/postcorr/byt5/retrain/.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support