Making Myanmar Spelling Book searchable

I was looking for a convenient way to save layers of text images to separate pdf files in the image processing sofware GIMP. Luckily, I found out that this could be done easily with Export Layers, a GIMP plug-in which is downloadable from here.

For a while I have been downloading old Myanmar related books in Myanmar or English language in pdf format from the Internet Archive. While I found these books to be “searchable”, they didn’t work well with searching for Myanmar language text. The obvious solution then will be to OCR the Myanmar language text and save the result (after correcting errors) to pdf format.
I have been trying out the Google Docs for applying OCR and found it quite dependable with pages extracted from old books in pdf format, or with images made with camera of my mobile phone. However, some pages in Myanmar language extracted directly from pdf files would not give editable text, or text at all with this method. I guess this might be due to the font used in producing the source document, because previously Myanmar language documents had been produced with a variety of non-Unicode propriety fonts. Obviously in such a situation, I could convert the pdf text into images and then apply OCR. This works!

Here, I’m sharing my experience in trying to convert an official Myanmar language spelling book into searchable pdf format.

The Source

The source is the Myanmar language spelling book (မြန်မာစာလုံးပေါင်းသတ်ပုံကျမ်း), available for download here:.
The following shows the cover of the book and a sample page of the text:

This is chosen to show my fellow dummies the usefulness of a basic facility such as a “text search” in a pdf document. In Myanmar language dictionaries, or a spelling book, I always find it hard to get to the right text. Quite hard, because the alphabetical order consisted of a complex system of consonant, additional consonants and vowels. And here, I am not the only exception because in the front matter of the Abridged Myanmar Language Dictionary, the Myanmar Language Commission acknowledged it so!

For my exercise I’m confining my material to cover the spellings lists only, that is, from page-1 to 285 of the book.

Oringinal work flow

I am using only the open-source or free software.

1. Extract pages 1-285 of the text (using PDFsam Basic software).
2. Convert the resulting pdf file to jpg images (using PDFMate PDF Converter software).
3. Convert resulting jpg images to pdf (using PDFMate PDF Converter software).
4. Merge the resulting pdf files as single pdf file (using PDFsam Basic software).
5. Open Google Drive; import the pdf file (29-MB); open in Google Docs. This converted the text-image to editable text. However, Google Docs would not process all of this information at one time and had to partition the input pdf file into three files.

Result

I had saved the Google Docs output as odt file. First two pages, opened in LibreOffice Writer are shown below. The OCR seems to be quite accurate and I’ve highlighted the single error on the first page. The format of the page is not very well preserved. But, as it is, it won’t distract you from searching text when scanning errors were corrected and the file has been converted to pdf format.

I tried reformatting one page to look like the original. Looks good, but too much work and I wouldn’t ever think of doing it for the whole file.

The next workflow

I knew how to import pdf files to graphic images layers in GIMP software, but couldn’t find how I could export each of them as a pdf file. Because of that I used PDFMate for converting pages of pdf file into graphic images. However, after a bit of Googling, I found that I could use Export Layers plug-in at the beginning of this post, I installed the plug-in and switched to this work-flow:

1. As above.
2. Open the resulting pdf file in GIMP using the Open as Layers… option in the File menu.Now I have all 285 pages of pdf opened as graphic layers.
3. By installing the Export Layers plug-in for GIMP, I now have additional options in the File menu. I select the Export Layers option and export the layers as separate pdf files.
4. As above.
5. As above.

Except for being more convenient for me, this approach produce exactly the same result as with the original approach.

Improving (hopefully) the format of the final pdf

From my earlier experiments, I found that if the two column pdf pages could be converted into one column pages before we send them to Google Docs for OCR operation, the OCR process could be smoother. And output text, I guess, would be something like a one-column format and look much neater than it was previously. So, I tried looking for preprocessing of the pdf file or the odt file derived from it to find some easy way to reduce two columns text to one colum. How ever it seems like too hard to do, even if ways to do that exist at all.

Then I tried looking for using programming (macros) in GIMP to do that for imported graphics. I was about to learn macro programming, when by luck I found out that images could be cut using the Slice Using Guides option in the Image menu of GIMP! Also, it works with image layers!

Left and right slices are exported as pdf files, one for each layer.

Editing the OCR results and converting them finally into pdf file

Now proceeding with steps-4 and 5 as with the original work-flow I got the same kind of results, but now in single column. After completing OCR with Google Docs, I saved it to an odt file. The format, alas, is still messy!

My idea is to open that file in LibreOffice Writer. I could then make neccessary corrections by referring to the original Spelling Book. Then convert it to pdf in LibreOffice. After converting all pieces of the Spelling Book to pdf, I could merge them using PDFSam. Note that Google Docs would process only about 80 pages of my graphic-pdf input and I had to feed my spelling book pdf in parts.

Well, after all that work, you are going to get a real pdf file in which you can search text. However, I didn’t do this last part, because I found a much simpler work-flow I’d overlooked. And it produces a neater output without extra work!

PDF2ODT or Text-image2OCR2ODT

In these somewhat boring days of social-distancing and my voluntary stay-at-home, I was glad to have two younger friends in their late sixties visiting me. They are still working and in the middle of our familiar idle chat I came to know that they were having a hard time typing out all the text in some standard formats in pdf documents in order to prepare new ones! So I showed them how to do that kind of thing by converting pdf to open document format. Or if the source documents were not in digital text format, to take photos of them and then convert them into text using OCR via Google Doc. Then to download them in odt format. I’m documenting these little tricks, lest they forget. I hope that somebody will improve on them.

1. True or Normal or Real PDF documents

These are made up of digitally created text. All you need to do here is to copy the text from these files and paste them in a blank page in a word processor. For example, you could Microsoft Word. If you don’t have a licensed Microsoft Word installed, I suggest the Open Source LibreOffice.

1.1. Copying text from PDF document

Here, I open the pdf file with Adobe Acrobat Reader and select all text. Typing the keys Ctrl+C copied them to clip board.

1.2. Paste text to a new document in LibreOffice

Create a new document in LibreOffice Writer and paste copied text. The fonts and text are Ok. All you’ll need to do is to format the heading, paragraphs, etc. You can do all the editing you need.

In Myanmar language text the original in pdf (left) copied and pasted (right) shows some minor character errors in addition to format of lines. The Myanmar text once pasted to the blank document should be changed to the original font (here: Pyidaungsu Myanmar Unicode).

1.3. Copying only a part of the PDF file, a paragraph, or only a few words

Adobe acrobat reader doesn’t allow copying only a part of the PDF file. If you want to do so a simple method is to open it in the Google Chrome browser and make your copy on the opened page.

1.4. How to extract only one or more page(s) from a true PDF document

You can use the open source software PDFsam. It can split and merge PDF files.

2. Scanned or image only documents in PDF or other formats

Most of the pre-Unicode digital documents in Myanmar language and scanned PDF documents or scanned text in other graphic formats fall in this category. This means they need to be converted to digital text through OCR (Optical Character Recognition) software.

2.1. First step: Create graphic image(s) of a page of text or a document

You could directly use OCR on text in scanned PDF document. It is probably fine for such text in English language. But my experience (limited) with Myanmar language shows that scanned PDF text converted to an image format (such as jpg or png) gives better result.
In this age of smartphones, taking the image of any printed text is easy and adequate for OCR. For creating image files I used GIMP the free and open source image editing software.

• for a scanned PDF file, you can directly open the file in GIMP; it will show separate pages of the file and you can select the page(s) to open in GIMP. You can edit and save each image. For example, I normally save them into png format.
• for printed documents you can take photos with your cell phone, import images to your PC, and process them with GIMP and create image files.

The following picture shows how I created a single colum text image (2) from a two-column source (1) to facilitate the OCR operation.

2.2. Second step: Upload the graphic image or file to GoogleDrive

For this you need to have a Google Account, and have the internet connection running. You open the GoogleDrive and upload your image file(s).

2.2.1. Click “New” in GoogleDrive

2.2.2. Upload image file(s)

Here I’ve uploaded my file in PNG format that have been created earlier with GIMP.

2.2.3. Running OCR on the imported image

Now I right click that file:

From the menu that appears, I select Open with/Google Docs. This causes Google Docs to run OCR on my image file. The image is just for one page of document. I think it will be wise to work page by page. That way, error checking and editing would be easier.

2.2.4. Download the completed file for editing with a word processor

Now the OCR has been completed. I’ve reduced the size of the text-image to be able to show it as well as the converted text below it. You could do the editing of the document here with Google Docs. But I prefer editing with a Word Processor after this file has been downloaded.

I prefer working with a free Open Source word processor, so I downloaded the completed file in ODT format.

After clicking the option to download the converted document, you will find your downloaded file in the “Downloads” folder. Now you can open it with LibreOffice or any other software that could handle the ODT file format.