this post was submitted on 28 Jun 2024
30 points (100.0% liked)

Open Source

31250 readers
269 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
30
Open source handwriting OCR? (hexbear.net)
submitted 4 months ago* (last edited 4 months ago) by [email protected] to c/[email protected]
9 comments fedilink hide all child comments
 

I'm looking for something that I can scan hand-written notes into and have OCR'd. Maybe one that I can even train on my handwriting. Ideally I end up with a searchable PDF of my notes.

People use one-note for this, but I'm not really comfortable with letting microsoft see my handwriting.

top 9 comments
sorted by: hot top controversial new old
[–] [email protected] 2 points 4 months ago (3 children)

To train an AI to recognize handwriting you need a huge dataset of handwriting examples. That is millions of samples of handwritten text + information about what the written text says in every example).

This is why the best engines only exists as a service in the cloud. The OCR engines you can install lovely that are acceptable, but far from perfect, are commercial. Parascript FormXtra is one of the better commercial ones.

The only OCR Engine that's free and really good is Tesseract OCR but it doesn't handle handwritten text.

[–] [email protected] 2 points 4 months ago

I don't really need the locally trained AI to recognize general handwriting, only my own.

I could provide a few pages of my own training data (maybe write out a few pages of "quick brown fox jumps over the lazy dog" and other stuff like that), and then ideally it flags stuff it's unsure about and I clarify some more. Maybe find garbled nonsensical sentences, realize it's probably a mistake, and try and fix it.

I assumed the leaps in AI would have taken care of this by now, since detecting handwritten letters from touch pen-strokes existed in the 90s. But I guess handing it a chunk of text is too different of a problem, instead of feeding it stroke by stroke?

[–] [email protected] 2 points 4 months ago (1 children)

Can you fine tune tesseract on a local hand writing dataset ? Or insert it in context like a pre-prompt ?

[–] [email protected] 3 points 4 months ago (1 children)

It wasn't possible a year ago when pos6ted around with tesseract. Things might have changed during the last couple of months though.

[–] [email protected] 1 points 4 months ago (1 children)

To train an AI to recognize handwriting you need a huge dataset of handwriting examples. That is millions of samples of handwritten text + information about what the written text says in every example).

then how can this model be so good? the dadaset is only 350 MB and the results seem insane ... sadly i have no idea how to use it.

[–] [email protected] 3 points 4 months ago* (last edited 4 months ago)

How good is good do you say?

We got a pretty good results with CER at 4% and WER at 15%!

This was on a limited dataset used to test and train which most likely means that if you introduced an even larger dataset with greater variations in handwriting style for testing the numbers might be even worse.

Very simplified: A risk of a character wrong every 20th character and a word wrong every 7th word. The SER was around 20%.

There's an reason why no one has released a good model for western letters yet and why companies pay up to 1€ for capturing data from 10 handwritten pages.

It will come but OCR isn't as sexy as developing text2image solutions.

[–] [email protected] 1 points 4 months ago

Rocket book has solid OCR. The app is free and there are templates available if you don't want to use the reusable paper.