Situation: I got a scanned book that I’d like to read that is in chinese and has no available translation. I really want to read it, because it would probably help a lot with my university project.
What I tried: tried creating a version with ocr to get a text layer and use some translation tool on it, but found no way to make the ocr text visible. I also tried this tool, but the ocr didn’t work for me, and I found no way to use it with some local model
Have any of you ever done a similar task? I’d appreciate any kind of suggestions and tips.
If you find that OCR doesn’t get you very far, maybe try a small vLM to parse PNGs of the pages. For example, Nanonets OCR will do this, although quite slow if you don’t have a GPU. It will give you a Markdown version of the page, which you can then translate with another tool.
PaddleOCR might also be useful, since it focuses on Chinese, but it’s more difficult to set up. To add to this, some other options are MinerU and MistralOCR (this is paid, but you can test it for free if you upload it in Mistral’s library).
That PaddleOCR looks very interesting. It will even extract images and formulas and somewhat preserve formatting in the output! I will try this one, even if takes more than a day to process is with my low end cpu. Thank you for the suggestion!
Be wary that their docs are so and so. Nanonets OCR, Mistral OCR and MinerU will also extract formulas and images.
One other model I forgot to mention is Docling. This one is quite quick to set up in a docker container, and will have a web interface ready to go where you can upload documents. This sort of follows the PaddleOCR pipeline, but also allows you to use vLMs.
Good luck!
This is more intended for real time usage, but might work for you:
https://github.com/Artikash/Textractor
https://github.com/Crivella/ocr_translate
I watch Macaw45 play full fledged Japanese retro RPG games using Textractor it’d probably be good for books too.
Thanks for the suggestions. That OCR_translate looks interesting. I will prioritize other recommended tools that seem to be more focused on books, but I bookmarked it for future needs.
i did this with a chinese book, but have to check what i used.
The translation was entirely readable.
i think i used tesseract.
No, GImagereader!
that was it.
tesseract was also very straightforward, but gimage reader had a GUI, and all I had to do was import the file and then click export and it did the whole thing.
I used tesseract, but the output pdf didn’t have visible text, and I found no way to change it. Maybe I don’t know how to properly use it., or it’s not intended to keep formatting.
try gImagereader.
it’s a frontend to tesseract and is more workable via its GUI and option menus.
Load the file, execute the program.
That’s all I had to do for a successful OCR.
You can literally just feed the images into chat gpt at this point.
I’m giving preference to open source tools, but that’s a good thing to know, thanks
Every time I’ve done it, it’s pretty bad. Ocr is much better.
This doesn’t work after the pdf reaches a cert max size.
Could just break it up into chapters or something, pretty easy to split a pdf.
Which Google lens work? And take a picture of each page and feed it to the Google translate engine. It might be the easiest way.
I’m not sure if it would be viable for a long book, and I’m also avoiding google, but thanks for helping. I got some nice suggestions in this thread.
notebooklm (Google)
Well, I’m avoiding google, but I will keep it in mind as a last last resort, thanks
