Tesseract (Screen OCR) UDF

[seangriffin]

This UDF provides text capturing support for applications and controls using Tesseract - an OCR engine currently developed by Google.

Tesseract was originally developed as proprietary software at Hewlett-Packard between 1985 until 1995. After ten years without any development taking place, Hewlett Packard and UNLV released it as open source in 2005. Tesseract is currently developed by Google and released under the Apache License, Version 2.0.

Tesseract is considered one of the most accurate free software OCR engines currently available. It was one of the top 3 engines in the 1995 UNLV Accuracy test.

My main goal in developing this UDF is to provide AutoIT users with a free Screen OCR solution that competes with other commercial (payed) technologies like Microsoft Office Document Imaging (MODI) and Textract.

REQUIREMENTS:

• AutoIt3 3.2 or higher
• Tesseract 2.01 or above

INSTALLATION:

To install Tesseract:

• Run the file http://web.aanet.com.au/seangriffin/Tesseract201.exe.
• Follow the installation instructions.

LIST OF FUNCTIONS:

_TesseractTempPathSet($temp_path)

_TesseractScreenCapture($get_last_capture = 0, $delimiter = "", $cleanup = 1, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

_TesseractWinCapture($win_title, $win_text = "", $get_last_capture = 0, $delimiter = "", $cleanup = 1, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

_TesseractControlCapture($win_title, $win_text = "", $ctrl_id = "", $get_last_capture = 0, $delimiter = "", $expand = 1, $scrolling = 1, $cleanup = 1, $max_scroll_times = 5, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

_TesseractScreenFind($find_str = "", $partial = 1, $get_last_capture = 0, $delimiter = "", $cleanup = 1, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

_TesseractWinFind($win_title, $win_text = "", $find_str = "", $partial = 1, $get_last_capture = 0, $delimiter = "", $cleanup = 1, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

_TesseractControlFind($win_title, $win_text = "", $ctrl_id = "", $find_str = "", $partial = 1, $get_last_capture = 0, $delimiter = "", $expand = 1, $scrolling = 1, $cleanup = 1, $max_scroll_times = 5, $scale = 2, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0, $show_capture = 0)

DEMONSTRATION:

<Under Construction>

EXAMPLES:

_TesseractControlCapture.au3_TesseractControlFind.au3

DOWNLOAD:

Latest Version - v0.6 (17/03/09)

Tesseract.au3

Edited May 21, 2010 by seangriffin

[ResNullius]

Nice work again seangriffin!

Note: For english recognition, also need to download http://tesseract-ocr.googlecode.com/files/...2.00.eng.tar.gz

Other language data files also available from http://code.google.com/p/tesseract-ocr/downloads/list

[seangriffin]

Thanks ResNullius.

I just began work on this. I needed a screen OCR capability for my Java UDF, so I can retrieve items from list boxes and combo boxes.

I did alot of research into OCR tools and found that Tesseract was well suited to AutoIT, because it's open source / free (I think) and it's good at OCR. Almost as good as Textract. I'm wondering how it's going to compare to ptrex's MODI script. I've only got MS Office 97 installed, so I don't have the MODI capability on my computer.

Hope to get a V0.1 UDF ready this weekend.

Edited February 13, 2009 by seangriffin

[AdmiralAlkex]

You should change "C:\Program Files\tesseract\tesseract.exe" to

@ProgramFilesDir & "\tesseract\tesseract.exe"

otherwise it won't work on localised versions of Windows

[seangriffin]

You should change "C:\Program Files\tesseract\tesseract.exe" to

@ProgramFilesDir & "\tesseract\tesseract.exe"

otherwise it won't work on localised versions of Windows

Thanks AdmiralAlkex. I'll definitely be doing that.

[seangriffin]

This UDF is now officially released.

Currently at V0.2.

AdmiralAlkex,

I've implemented your change.

ResNullius,

I've replaced the link to the Tesseract download with an installer I created. Should stop people installing Tesseract in the wrong place for my UDF to work.

[seangriffin]

V0.3 released.

Main change is the addition of a $show_capture parameter, to make it easier to debug and adjust the parameters of the _TesseractCapture function if text recognition is not accurate.

[seangriffin]

V0.4 released.

Added the function _TesseractFind to allow searching for text within recognised text.

[longfields]

This is a very welcome addition - thanks very much.

However, its a pain that you can only extract text from a rectangle within a window if you have a control ID. Some proprietary programs don't provide control IDs and so you have to scrape the whole window. That's much better than nothing, but I'd still like to be able to target a particular area.

Is there any way to change this?

Once again, many thanks!

[BinaryBrother]

Awesome script... I was just looking through some stuff and was about to purchase one!! Then I was like...

"Let me go check the AutoIt forums real quick... Just to see..."

Your a life saver... I almost bought that SimpleOCR SDK....

Edited March 8, 2009 by BinaryBrother

[m112358]

Hey..

Im trying to make it work, but i keep getting an error message.

When i run the _TesseractCapture it opens the IE window and opens up some gui window, but then it pops up with the message that there is an error in line 221, gives me the gode

Line 221 (File "C:\Program Files (x86)"\AutoIt3\Include\Tesseract.au3)

$Obj1.ShowFile($capture_filename,1)

$Obj1^ERROR

Error: Variable must be of type "Object".

Ive checked if the paths to the tesseract exe is correct in the tesseract.au3 file and everything looks good, but of some reason its not working.

Is there anyone that could explain to me why?

Oh and btw, im running Vista Home Premium 64bit. Dunno if that makes a difference?

Malakith

[seangriffin]

V0.6 released.

The function "_TesseractCapture" has been split into the following 3 functions:

• _TesseractScreenCapture
• _TesseractWinCapture
• _TesseractControlCapture

The function "_TesseractFind" has been split into the following 3 functions:

• _TesseractScreenFind
• _TesseractWinFind
• _TesseractControlFind

Also added a "_TesseractTempPathSet" function, to allow the temporary path to be changed to something other than "C:\". In case users without write access to "C:\" are having troubles with the UDF.

longfields feedback reminded me of how focused the UDF was on controls, and not so much on windows or the entire screen. The existing two functions were already overloaded, so for simplicity I decided to split both into 3 seperate functions each, covering the screen, windows and controls. Let me know what you think longfields.

This was a quick release so I haven't done much testing. Let me know if you find any issues.

Edited March 17, 2009 by seangriffin

[seangriffin]

Hey..

Im trying to make it work, but i keep getting an error message.

When i run the _TesseractCapture it opens the IE window and opens up some gui window, but then it pops up with the message that there is an error in line 221, gives me the gode

Line 221 (File "C:\Program Files (x86)"\AutoIt3\Include\Tesseract.au3)

$Obj1.ShowFile($capture_filename,1)

$Obj1^ERROR

Error: Variable must be of type "Object".

Ive checked if the paths to the tesseract exe is correct in the tesseract.au3 file and everything looks good, but of some reason its not working.

Is there anyone that could explain to me why?

Oh and btw, im running Vista Home Premium 64bit. Dunno if that makes a difference?

Malakith

Hi Malakith,

Yes I think anyone with Vista that uses the "$show_capture = 1" parameter will get this error. This code came from another AutoIT forum topic. The code accesses the "Windows Image and Fax Viewer" object that comes with Windows XP. I use this because it supports the TIFF files generated by Tesseract. I believe the "Windows Image and Fax Viewer" has been removed from Windows Vista, and this is why you are getting the error. I use Windows XP myself. If anyone has a TIFF viewer solution for both Win XP and Vista I'd like to know!

Edited March 17, 2009 by seangriffin

[longfields]

seangriffin,

Many thanks for your work on this - I'm starting to get some results. A couple of observations:

1. Accuracy of OCR significantly deteriorates if you scan a very small part of the screen. I expect that this is an issue with Tesseract rather than with the UDF.

2. If you're scanning an area of the screen and then change the scale, a different part of the screen is then scanned. This doesn't seem very logical, but there may be a good reason for it.

3. It looks like there are things you can do to train tesseract - I haven't gone into this in detail yet.

This looks really promising - thanks again.

[seangriffin]

Thanks longfields,

Yeah it's very early days for the UDF. There's a long way to go I think. One step at a time. Thanks for your feedback...

1. Accuracy of OCR significantly deteriorates if you scan a very small part of the screen. I expect that this is an issue with Tesseract rather than with the UDF.

I haven't experienced this myself. I've mainly focused on window controls, and specifically list boxes and combo boxes. These are quite small but probably not as small as the area you are dealing with. Could you send me an example? I'd like to see the problem for myself.

2. If you're scanning an area of the screen and then change the scale, a different part of the screen is then scanned. This doesn't seem very logical, but there may be a good reason for it.

I thought this problem might occur. Yes I think my image resizing script has bugs. I have only done some quick checks, and to my eye it seemed to be working. The problem would be in my function "CaptureToTIFF", and I think my use of "_WinAPI_CreateCompatibleBitmap" and "_GDIPLus_GraphicsDrawImageRect" is not quite right. This is where I resize and reposition the screen capture using the 4 indents. I'm actually on the lookout for a better resizing script. I feel my script blurs / softens the screen capture too much after it's repositioned and resized, and Tesseract then has a hard time recognising the text in this modified image. Tesseract is definitely more accurate when images are larger in size, though the image needs to remain clear. My use of "_WinAPI_CreateCompatibleBitmap" and "_GDIPLus_GraphicsDrawImageRect" does not produce a clear enough image.

3. It looks like there are things you can do to train tesseract - I haven't gone into this in detail yet.

Yes you're right, Tesseract can be trained. I've never tried either.

Edited March 19, 2009 by seangriffin

[Authenticity]

Amazing recognition and pretty fast I must say. Thank you very much for the share.

[trinitrotoluen]

thank you seangriffin ! Great UDP !

[3xM3NT4Lx4]

To anyone that understands this, can you please refer to my thread?

I need help with my script.

http://www.autoitscript.com/forum/index.php?showtopic=92291

[hendrikhe]

To anyone that understands this, can you please refer to my thread?

I need help with my script.

http://www.autoitscript.com/forum/index.php?showtopic=92291

Maybe my thread can help you:

http://www.autoitscript.com/forum/index.php?showtopic=92795

I modified a bit the script there to use _TesseractScreenCapture like other AutoIt standard pixel-functions

PixelChecksum or Pixelsearch:

_TesseractScreenCapture($get_last_capture = 0, $delimiter = "", $cleanup = 1, $scale = 2, $iLeft = 0, $iTop = 0, $iRight = 1, $iBottom = 1, $show_capture = 0)

Really an amazing work seangriffin, I would have never resolved my problem without ur script.

Best regards

Hendrik

[goldenix]

REQUIREMENTS:

Java runtime is required?

Nothing happens if I run the sample, it just opens IE page & thats it, it just sits there. i do not have java.