Ancient Greek OCR

cwconrad
Posts: 2112
Joined: May 5th, 2011, 5:52 pm
Location: Burnsville, NC 28714
Contact:

Ancient Greek OCR

Post by cwconrad »

In an AWOL blogpost by Nick White (http://ancientworldonline.blogspot.com/ ... k-ocr.html), what may well be a splendid new tool for digitizing out-of-print materials with Greek texts:
Ancient Greek OCR is free software to accurately convert scans of printed Ancient Greek into unicode text and PDF files, which can be easily searched, copied, archived, and transformed. It uses the excellent Tesseract OCR engine, tailored for Ancient Greek typography, syntax and vocabulary.

It works with Windows, OS X, Linux and Android, and works on personal computers, mobile devices, and large server clusters.
οὔτοι ἀπ’ ἀρχῆς πάντα θεοὶ θνητοῖς ὑπέδειξαν,
ἀλλὰ χρόνῳ ζητέοντες ἐφευρίσκουσιν ἄμεινον. (Xenophanes, Fragment 16)

Carl W. Conrad
Department of Classics, Washington University (Retired)
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Ancient Greek OCR

Post by Jonathan Robie »

I asked Bruce Robertson about the relationship between this and his Lace OCR work. He sent me this response, wiht permission to post it:
Bruce Robertson wrote:We are all working together on Greek OCR, using different engines and eventually combining results to get the best possible output.

Nick's engine ('Tesseract') is the only one that right now makes sense for people to try out on the desktop (unless they have linux experience). In contrast, I'm working on high-volume HPC solutions that ought, and usually do, eek out a couple more percent accuracy. There are differences in the fonts trained and how language models figure into things. I would recommend that someone wanting to work on the files already available in our Lace repository http://heml.mta.ca/lace/catalog download the corresponding zip or tar file from there. If you want a text in archive.org but not available through Lace, try it out with Nick's Tesseract engine, but also let us know so we can all gauge public interest. Ultimately, it will be helpful to scholarship in general if we create a comprehensive morphologically aware repository, rather than a bunch of small personal collections, though the latter is the only legal option for copyrighted materials.
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
Peter Pankonin
Posts: 31
Joined: May 31st, 2011, 11:18 am
Location: Lethbridge, Alberta, CANADA

Re: Ancient Greek OCR

Post by Peter Pankonin »

I asked Nick about OCR in documents that contain both English and Greek (or other languages), like Lexicons, and he posted a useful tip here:
http://ancientgreekocr.org/windows.html#tips
--
Πέτρος Πανκώνιν
"There are 10 types of people in the world, those who understand binary and those who don't"
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Ancient Greek OCR

Post by Jonathan Robie »

I decided to compare output from Tesseract and Nick White's support for polytonic Greek to output from Bruce Robertson's Lace. Tesseract is a lot easier to install and runs with more reasonable resources, Lace is designed for higher end computers and uses fancier processing to combine results scanned using different algorithms.

The image is the same for both. I have attached it to this page:

Here's Tesseract and Nick White's Greek support:
Tesseract wrote:Καὶ ἐκ τοῦ καπνοῦ ἐξῆλθον ἀκρίδες.

Ἡ μνήμη τῶν ἀκρίδων, ὖποσημαίνειν ἕνεκεν οἷπηνεστέραν τῆν
μνῆμην τῆς τῶν οἱμοιρτωλῶν ἐν τῇ προσκαῖρφ αὗτοῢ ζανῇ ἐκτοπω-
τοὶτης διαγωγῆς. διὶ καὶ ἔκ τοῦ καπνοῖι τῆν -γένεσῖν φησιν
ἐσχηκέναι. ἒπειθτερ καὶ τῶν αἷσθητῶν οἰκρῖδων ᾖ γένεσις ἐκ,

υυ ʹ Ι Ὼ 3 Ι 9 Ὼ Ὼ Ὼ
ψθοροποιου περιεχοντος λαμβανει την αφορμην. αλλα και το
ἆιιρῖδεδς ὄνομα, οὖκ ἔξω τοῦ' κατζᾶκρακς τὶις τιμωρῖας ἒποιχθ-'ἧναι

Ι ʹ Ρ 2 ʹ η

προς ηλωσιν παρεληφθη. οσον γαρ το ακολαστον των ἁμαρτη-
σοὶντων τῇ μνήμῃ παρίσταται, τοσοῦτον κα) ᾖ ὄργῆ τοῢ λαχόντος
κρίνειν ἒξσξπτεϊαι μυσσοιξδσέἷ ἀεὶ ,ξπαρσνσῖᾳι τσῦ Ὼτξϙν αἶγα-
νακτησιν αναφλεγοντος. ε οσθαι ε ε ουσιαν ως επι εμψυχων
αὗταῖς διεσκευᾶσθη' τῇ προσωποποιΐᾳ, τῴ' οἷνυποστόλως καὶ
ἆπαρεμποδίστως' τᾶς τιμωρῖας ἀνύεσθαι. ἇλλ' 'ἵνα μῆ τις τᾶ
τῶν ἇκρίδων δήγματα εἰδῶς οι) τοσαῦτην ὄὸὔνην παρέχοντα, φο-
ρηι-ᾗν ὖπολοὶβοι τῇν ἀλγηδόνα, τὸ ζι-ῶν σκορπίων ἒπιφέρει ὕπο'-
δειγμα, ἆλγεινοτέραν τὴν αἴσθησιν τῇ τοῦ κὲπρου ἒγχριᾷιψει
παρεχόντων. διὸ καὶ ἒρῤέθη οὖ τοῖς σκορπῖοις ἇλλδι ταῖς ἀκρίσι,
τῇ προσωποτοιΐᾳ ἒπιμὲνων κᾲνταῦθα ;, ῗνα μῇ ἆδικῆσωσιν οὗ τοῦς
ἇνθρόπονς, ἆλλὶι τὶι ἆπὸ γῆς φυοζιιενα. ''καὶ τινδς χᾶριν ἑξῆςᾶῢῢ*
'ἔχουσιν ἔξουσῖαν κατδι τῆν δοθεῖσαν δηλονότι, οὖ ϊοῗς σκορπῖοις
κ 9 1 2 χ ω ς 1 λ 1 ’ , χ ’ .
γαρ εδοθη αλλα ταις ακρισι' και τινος εξουσιαζειν τας ακρῖδαςη
, τῆς γῆς' οι' σκορπῖοι δὲ παρελεῖφθησαν οὔ μόνον εἷς 'ἔμφασιν
τοῖ') δραστικῶτεραν τῆς τῶν ἇκρίδων 'δήξεως τῆν αἴσθησιν τὸ
τῦμμα τοῢ σκορπίοι) ἐμποιεῖν, ἆλλ' ὃʹϊι καὶ τῷ οὖραίῳ πλήττων
ὄ σκορπῖος τῆν εἷς ὖστερον ἇποκειμένην αἷνίττεται τοῖς κακο-
τοιοῗς ἒπωδῦνην ἄλγησιν. ει' 'δὲ μῆ σαφῶς οῧτως ο' τῆς Ἁποκα-
λῦψεως λόγος διέξεισι, προεῖρηται πολλοὶκις ὧς ἐστ' ἕλαττον τῆς
ειΞρημοδσῺι-νηςῺῆ Θεοῢ Ἑκκλησία πεφρόντικε τῆς ψυχικῆς σωτη-
ριας' ιο και Παῦλος ο' θεῖος φησὶν, '; ἔχομεν τδν θησαυρὶν τοῦ-
'' ϊον ἒν ὄστρακίνοις σκεύεσι.» θησαυρὸν, ἕν ᾦ ὁ πλοῦτος τῆς τοῦ,
Χριστοῦ οἰπόκειται συγκαταβάσεως σοφίας' ἧ σοφῖα οὗ ῥητο,
ρικαῖς περιείργεται φληναφίοιις, ἆλλὶι παιδευτικῶν σννθὲσεων
ἐξηγήσει' καὶ δῆλοι) τοῧτο ἆπ' αὗτῆς τῇς προκειμὲνης χρήσεως'
Definitely useful, and it saves you a lot of typing, but it requires a fair amount of hand editing.

Here's Lace for the same image:
Lace wrote:Καὶ ἐκ τοῦ καπνοῦ ἐξῆλθον ἀκρίδες.
Ἡ μνήμη τῶν ἀκρίδων, ὑποσημαίνειν ἕνεκεν ἀπηνεστέραν τὴν
μνήμην τῆς τῶν ἁμαρτωλῶν ἐν τῇ προσκαίρῳ αὐτοῦ ζωῇ ἐκτοπω-
τάτης διαγωγῆς. διὸ καὶ ἐκ τοῦ καπνοῦ τὴν γένεσίν φησιν
ἐσχηκέναι . ἐπείπερ καὶ τῶν αἰσθητῶν ἀκρίδων ἡ γένεσις ἐκ
φθοροποιοῦ περιέχοντος λαμβάνει τὴν ἀφορμήν. ἀλλὰ καὶ τὸ
ἀκμες ὄνομα, οὐκ ἔξω τοῦ κατ' ἄκρας τὰς τιμωρίας ἐπαχθῆναι
πρὸς δήλωσιν παρελήφθη. ὅσον γὰρ τὸ ἀκόλαστον τῶν ἁμαρτη-
σάντων τῇ μνήμῃ παρίσταται, τοσοῦτον καὶ ἡ ὀργὴ τοῦ λαχόντος
κρίνειν ἐξάπτεται μυσσούσης ἀεὶ τῇ παρουσίᾳ τοῦ τὴν ἀγα- lo
νάκτησιν ἀναφλέγοντος. δεδόσθαι δὲ ἐξουσίαν ὡς ἐπὶ ἐμψύχων
αὐταῖς διεσκευάσθη "* τῇ προσωποποιίᾳ, τῷ ἀνυποστόλως καὶ
ἀπαρεμποδίστως ^ τὰς τιμωρίας ἀνύεσθαι. ἀλλ' ἵνα μή τις τὰ
τῶν ἀκρίδων δήγματα εἰδὼς οὐ τοσαύτην ὀδύνην παρέχοντα, φο-
ρητὴν ὑπολάβοι τὴν ἀλγηδόνα, τὸ τῶν σκορπίων ἐπιφέρει ὑπό.
δεῖγμα, ἀλγεινοτέραν τὴν αἴσθησιν τῇ τοῦ κέντρου ἐγχρίμψει
παρεχόντων. διὸ καὶ ἐρρέθη οὐ τοῖς σκορπίοις ἀλλὰ ταῖς ἀκρίσι,
τῇ προσωποποιίᾳ ἐπιμένων κἀνταῦθα *, ἵνα μὴ ἀδικήσωσιν οὐ τοὺς
ἀνθρώπους, ἀλλὰ τὰ ἀπὸ γῆς φυόμενα. uκαὶ τινὸς χάριν ἑξῆς····
ἔχουσιν ἐξουσίαν κατὰ τὴν δοθεῖσαν δηλονότι, οὐ τοῖς σκορπίοις
^ '?> '/1 ' r ^ ^ ~ ' ' . » / >S- /v V » /5«
γὰρ ἐδόθη ἄλλα ταῖς ἄκρισι καὶ τινος ἐξουσιάζειν τὰς ἀκρίδας ;
τῆς γῆς· οἱ σκορπίοι δὲ παρελείφθησαν οὐ μόνον εἰς ἔμφασιν
τοῦ δραστικωτέραν τῆς τῶν ἄκμων δήξεως τὴν αἴσθησιν τὸ
τύμμα τοῦ σκορπίου ἐμποιεῖν, ἀλλ' ὅτι καὶ τῷ οὐραίῳ πλήττων
ὁ σκορπίος τὴν εἰς ὕστερον ἀποκειμένην αἰνίττεται τοῖς κακο-
ποιοῖς ἐπωδύνην ἄλγησιν. εἰ δὲ μὴ σαφῶς οὕτως ὁ τῆς Αποκα-
λύψεως λόγος διέξεισι, προείρηται πολλάκις ὡς ἐπ' ἔλαττον τῆς
εὐρημοσύνης ἡ Θεοῦ Ἐκκλησία πεφρόντικε τῆς ψυχικῆς σωτη-
ρίας· διὸ καὶ Παῦλος ὁ θεῖος φησὶν, " ἔχομεν τὸν θησαυρὸν τοῦ·
" 'τον ἐν ὀστρακίνοις σκεύεσι.’' θησαυρὸν, ἐν ᾧ ὁ πλοῦτος τῆς τοῦ
Χριστοῦ ἀπόκειται συγκαταβάσεως σοφίας· ἡ σοφία οὐ ῥητο-
ρικαῖς περιείργεται φληναφίαις, ἀλλὰ παιδευτικῶν συνθέσεων
ἐξηγήσει· καὶ δῆλον τοῦτο ἀπ' αὐτῆς τῆς προκειμένης χρήσεως·
It also needs editing, but less.

I'd be interested in seeing how other OCR packages do with this image. Any takers?
Attachments
catenaegraecorum08cramuoft_0325.jpg
catenaegraecorum08cramuoft_0325.jpg (320.38 KiB) Viewed 14036 times
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
BruceRobertson
Posts: 22
Joined: May 16th, 2014, 9:49 pm

Re: Ancient Greek OCR

Post by BruceRobertson »

Hi, all. Jonathan has been very kind in promoting Nick's work and mine here. The following is very much 'Inside Baseball'. If you aren't interested in the details of Greek OCR, just be aware that this field is shifting rapidly and improvements are being made monthly. OCR is never perfect, but we are getting many results on very difficult texts that are way better than even a year ago. See, for instance, Migne vol. 45:
http://heml.mta.ca/lace/sidebysideview2/11925039
The scan is pretty atrocious, but we get some pretty good output.

If you want the details, then a quick word about terminology. 'Lace' is the name I'm using for the website and related code which allows us to view facing page images and html OCR output. The code for this is in github https://github.com/brobertson/Lace, and our site is at http://heml.mta.ca/lace. It currently has 780 books in it, all generated with resources from Compute Canada.

The output of any OCR engine that generates html can be viewed in Lace. Tesseract is one such engine. Nick, now working with the Perseus group in Boston, is a whiz at Tesseract, initially training it as a side project to a larger effort in the UK!

Gamera is another engine. Federico Boschetti and I started working with it a couple of years ago and last year built an entire HPC stack around eeking out the best OCR from Gamera and the OCR that archive.org already provides. This stack, called Rigaudon, includes automatic spellcheck and OCR output combining. It was with Gamera/Rigaudon that the first 500-600 books in the Lace repository were generated. Running Rigaudon at home requires a multi-core machine and some pretty sophisticated software.

Another promising engine is Ocropus 0.7. Its basic recognition is very impressive, it can be trained fairly simply, and it is able to deal with a variety of letter thicknesses, etc. One downside is it requires much training data; however, we can use the data from Gamera/Rigaudon to train it! I've been focusing on this since about January, making classifiers for Porson and Teubner fonts, and moving on from that. To get the very best results, though, we need to do all the additional things that Rigaudon provided to Gamera. The complete Ocropus stack for polytonic Greek is called Ciaconna, and like the rest is available in github.

Ultimately, we will use all three engines in harmony with each other. Federico wrote a paper on this in, I think 2008, so we know how to do it. But right now, it makes sense to work up each engine separately. We are focusing on these because they are free and open-source.
Louis L Sorenson
Posts: 711
Joined: May 5th, 2011, 9:21 pm
Location: Burnsville, MN, USA
Contact:

Re: Ancient Greek OCR

Post by Louis L Sorenson »

Here is an update by Nathaniel J Erickson on ntgreeketal.com as of May 14, 2020: https://ntgreeketal.com/2020/05/14/ocr- ... -surprise/
Eeli Kaikkonen
Posts: 611
Joined: June 2nd, 2011, 7:49 am
Location: Finland
Contact:

Re: Ancient Greek OCR

Post by Eeli Kaikkonen »

Don't be fooled to skip to the conclusion, because in the middle of that article is an update with a link to a later post:

https://ntgreeketal.com/2021/03/27/ocr- ... tesseract/
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Ancient Greek OCR

Post by Jonathan Robie »

Eeli Kaikkonen wrote: February 5th, 2022, 5:09 am Don't be fooled to skip to the conclusion, because in the middle of that article is an update with a link to a later post:

https://ntgreeketal.com/2021/03/27/ocr- ... tesseract/
Yes.

And for perspective, I don't think that modern Greek OCR worked well back when Bruce Robertson started his work on Lace. Back then, nothing worked all that well, so this was a breakthrough. And it's a big part of how Perseus was able to digitize so much Greek.

But I'm curious. I attached an image to an earlier post, giving results for two systems. Can anyone post results for current systems using the same page?
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
Eeli Kaikkonen
Posts: 611
Joined: June 2nd, 2011, 7:49 am
Location: Finland
Contact:

Re: Ancient Greek OCR

Post by Eeli Kaikkonen »

Jonathan Robie wrote: February 5th, 2022, 6:20 pm
But I'm curious. I attached an image to an earlier post, giving results for two systems. Can anyone post results for current systems using the same page?
The old attachment doesn't work.
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Ancient Greek OCR

Post by Jonathan Robie »

Oops! Let me upload it again.

I'd love to see OCR results from various tools using this image - anyone want to show results from the tools they prefer? Can you do better than the Lace results shown above?

I think this is a cleaner image than what I had before, so it's not an entirely fair comparison, but I am not sure I can find exactly the same image again.

08.CatenGraecPatr.NT.CatEpis.Jm.to.Apoal.v8.Cramer.1844._0325.jpg
08.CatenGraecPatr.NT.CatEpis.Jm.to.Apoal.v8.Cramer.1844._0325.jpg (181.58 KiB) Viewed 3324 times
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
Post Reply

Return to “Unicode and Fonts”