OMR — image → MusicXML, training status
From-scratch CNN+Transformer (7.6M params) on MetaScore/PDMX staff renders.
Snapshot
… ·
e7macPipeline jobs
loading…
Where the model stands (20k-staff scale, 192-sample val)
| Model | Clean SER ↓ | Clean exact | Warped SER ↓ | Warped exact |
|---|
| VLM + LoRA (exp8, abandoned) | 0.72 (norm-ed) | 0% | — | — |
| Phase-2, clean-trained (staff_omr_clef) | 0.158 | 24.5% | 0.727 | 1.6% |
| Phase-2 + scan augmentation (staff_omr_aug) | 0.162 | 24.5% | 0.188 | 17.2% |
| homr off-the-shelf (reference) | ~0.30 page-level | — |
Warped = skew / elastic warp / blur / contrast / noise applied to val images
(eval_aug.py, fixed seed). Augmentation closed the scan-robustness gap
(0.727 → 0.188) at zero cost to clean accuracy. Remaining gap to "usable":
exact-match must climb from ~25% toward 60%+; that's what the 50× data scale-up is for.
Training curves
staff_omr_aug — augmented fine-tune (stopped early, step ~5.9k)
staff_omr_v1 — original 60-epoch from-scratch run
What the augmentation looks like
Val staves with skew / warp / blur / contrast / noise — the model trains on these
Staff-level examples (held-out val, best checkpoint)
loading…
Full-page demos (segment → transcribe → stitch → MusicXML)
Clean piano page — works. 6 grand-staff systems, mostly-correct key/time/pitches.
input
model output (rendered)
High-res real page (Hallelujah) — fails instructively. Segmentation is correct
(6 systems found), but the model reads the bass staff as rests and drops time signatures.
This is the failure mode the 1.1M-staff run is meant to fix.
input
model output (rendered)
Low-res single-staff page — weak. Model hallucinates grand-staff structure on
single-staff input (s≈8px lines, below the training distribution).
input
model output (rendered)
