Recognizers for the Livro de Registro de Leis nº 1 (São Carlos, 1892–1894)
Kraken line recognizers fine-tuned from PP-OCRv6 medium (Kiessling, Zenodo 10.5281/zenodo.21788410) on 19th-century Brazilian Portuguese handwriting. They are used by the pipeline at DeguShi/OCR-pipeline-historical-data.
| file | version | SHA-256 |
|---|---|---|
st_c1m.safetensors |
v6 (final) | 8b1488d66916357ab9ad1770e5e53777573295f5d2bbc987fb25c717a75ad15a |
st_g3m.safetensors |
v5 | d41ac347a293d5d7a3588a7a0b771aa7a9332f3a540b13f7ded18f11fd1eee4d |
Training
- Data: line crops from 78 pages of the same register. The labels are diplomatic transcriptions produced with Gemini 4 Argon and aligned to the segmented lines. They were not reviewed by a person.
- Never seen in training: the 12 pages with an official transcription and the 10 test pages.
- Setup (st_c1m): 20 epochs, learning rate 1e-4, augmentation, seed 42, one NVIDIA L4.
Results (full pipeline, v6)
The numbers below come from the whole pipeline, which adds a period dictionary and a trigram context model; they are not the recognizer alone. On 3 held-out pages, against the official transcription: WER 18.74%, CER 4.43%. On 13 pages outside training and selection, against the diplomatic reference: WER 4.10%, CER 1.68%.
Use
git clone https://github.com/DeguShi/OCR-pipeline-historical-data.git && cd OCR-pipeline-historical-data
pip install -r requirements.txt && python setup_pipeline.py --skip-install # downloads and checks st_c1m
python -m ocr_pipeline.batch --config configs/final_v6.json --images my_pages/ --out outputs/batch
Limitations
The recognizers were trained on a single hand and a single book. They also learned the notation of their labels,
for example 1.º for ordinals. The source images belong to the Fundação Pró-Memória de São Carlos.
Post-correction baseline: ByT5-small (postcorr_byt5_small/)
Baseline C of the post-correction comparison (not used by the pipeline). google/byt5-small trained to correct
st_c1m's raw greedy line text, on 80k synthetic pairs: clean lines from the 78 training-page transcriptions and
ALESP laws 1889–1910, corrupted with character substitution/deletion/insertion statistics estimated on the 9 tuning
pages. No held-out or test page was used. 1 epoch, batch 16 × 2, lr 5e-4, max 128 bytes, seed 42, one T4
(3,395 s), transformers 4.51.3 (env.txt holds the full pip freeze).
model.safetensors SHA-256: cf52bdeeddc4cdc166afb745f3c462d1fd4ba903fa269af66d380563c2cbf078.
This is a retrain with the same code, data and seed as the run reported in the paper (whose weights were lost);
GPU training is not bit-exact. WER of this checkpoint (beam 4): held-out 21.89%, test 6.81% (paper run: 21.26%,
6.86%; v6: 18.74%, 3.49%). Code: tools/baseline_byt5.py; records: records/postcorr/byt5/retrain/.