Leaderboard

first of all i would like to thank @Danyfirex for this wonderful UWPOCR UDF that he offers us I noticed that when you perform OCR from a small area of the screen, it doesn't recognize it normally and can't read from it. and so I proceeded to these functions. Which work as I expected, However, I have no experience with GDIPlus. I post them, to share it with the community, and to get some hint, advice. In the example below I'm targeting the date on the bottom right of the taskbar, I have a 1920*1080 screen and 100% scale ; https://www.autoitscript.com/forum/topic/211521-ocr-from-a-small-area/?do=findComment&comment=1530475 #AutoIt3Wrapper_Run_Debug_Mode=Y #include <GDIPlus.au3> #include <GUIConstantsEx.au3> #include <ScreenCapture.au3> #include <WindowsConstants.au3> #include "UWPOCR.au3" ; * <-"https://www.autoitscript.com/forum/topic/207324-uwpocr-windows-platform-optical-character-recognition-api-implementation" _Example() ;-------------------------------------------------------------------------------------------------------------------------------- Func _Example() Local $sImageResult, $sOCRTextResult ; 1) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; From my monitor capture the date of the tskbar = 1808, 1062, 1862, 1074 $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074) ConsoleWrite("$sImageResult=" & $sImageResult & @CRLF) ; reading text from $sImageResult $sOCRTextResult = _UWPOCR_GetText($sImageResult) ConsoleWrite("- 1) OCR Result=" & $sOCRTextResult & @CRLF) ShellExecuteWait($sImageResult) ; 2) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; capture the date and give a border with 20 pixels = 1808, 1062, 1862, 1074, 0, 20 $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074, 0, 20) ; reading text from $sImageResult $sOCRTextResult = _UWPOCR_GetText($sImageResult) ConsoleWrite("- 2) OCR Result=" & $sOCRTextResult & @CRLF) ShellExecuteWait($sImageResult) ; 3) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; capture the date, give a border with 20 pixels, and invert colors = 1808, 1062, 1862, 1074, 1, 20 $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074, 1, 20) ; reading text from $sImageResult $sOCRTextResult = _UWPOCR_GetText($sImageResult) ConsoleWrite("- 3) OCR Result=" & $sOCRTextResult & @CRLF) ShellExecuteWait($sImageResult) ; 4) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ; and the same from memory ConsoleWrite("Delete tmp_OCR_image=" & FileDelete($sImageResult) & @CRLF) $sOCRTextResult = _GetText(1808, 1062, 1862, 1074, 1, 20) ConsoleWrite("- 4) OCR Result=" & $sOCRTextResult & @CRLF) EndFunc ;==>_Example ; #FUNCTION# -------------------------------------------------------------------------------------------------------------------- ; Name...........: _ScreenCapture ; Description ...: Captures a region of the screen ; Syntax.........: _ScreenCapture($sFileName [, $iLeft = 0 [, $iTop = 0 [, $iRight = -1 [, $iBottom = -1 [, $iNegative = 0 [, $iBorder = 0 [, $dScale = 1 [, $iBrightness = 0 [, $iContrast = 0]]]]]]]]]) ; Parameters ....: $sFileName Full path and extension of the image file ; $iLeft [optional] X coordinate of the upper left corner of the rectangle ; $iTop [optional] Y coordinate of the upper left corner of the rectangle ; $iRight [optional] X coordinate of the lower right corner of the rectangle. If this is -1, the current screen width will be used ; $iBottom [optional] Y coordinate of the lower right corner of the rectangle. If this is -1, the current screen height will be used. ; $iNegative [optional] 1 = Negative color, 0 = Normal color ; $iBorder [optional] Draw a border araunt, The color is taken from first pixel ; $dScale [optional] Scale factor ; $iBrightness [optional] Integer in the range -255 through 255 that specifies the brightness level. ; $iContrast [optional] Integer in the range -100 through 100 that specifies the contrast level. ; Return value...: Filepath of the image If the image is successfully saved. ; False If the image is Not successfully saved. ; ; Author ........: ; Notes .........: ;-------------------------------------------------------------------------------------------------------------------------------- Func _ScreenCapture($sFileName, $iLeft = 0, $iTop = 0, $iRight = -1, $iBottom = -1, $iNegative = 0, $iBorder = 0, $dScale = 1, $iBrightness = 0, $iContrast = 0) Local $hHBitmap, $hBitmap, $hGDIPlusBitmap, $hImage, $hImageCtxt, $vRet, $iBmpW, $iBmpH, $iBorderColor _GDIPlus_Startup() $hHBitmap = _ScreenCapture_Capture("", $iLeft, $iTop, $iRight, $iBottom, False) If @error Then Return SetError(1, 0, False) Local $tSIZE = _WinAPI_GetBitmapDimension($hHBitmap) $iBmpW = $dScale * DllStructGetData($tSIZE, 'X') $iBmpH = $dScale * DllStructGetData($tSIZE, 'Y') ;Default ;$iFlags=0,$iIlluminant=0,$iGammaR=10000,$iGammaG=10000,$iGammaB=10000,$iBlack=0,$iWhite=10000,$iContrast=0,$iBrightness=0,$iColorfulness=0,$iTint=0 Local $iIlluminant = 0, $iGammaR = 10000, $iGammaG = 10000, $iGammaB = 10000, $iBlack = 0, $iWhite = 10000, $iColorfulness = 0, $iTint = 0 Local $tAdj = 0 $tAdj = _WinAPI_CreateColorAdjustment($iNegative, $iIlluminant, $iGammaR, $iGammaG, $iGammaB, $iBlack, $iWhite, $iContrast, $iBrightness, $iColorfulness, $iTint) $hBitmap = _WinAPI_AdjustBitmap($hHBitmap, $iBmpW, $iBmpH, $HALFTONE, $tAdj) If @error Then Return SetError(2, 0, False) $hGDIPlusBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBitmap) ; Add Border If $iBorder > 0 Then $iBorderColor = _GDIPlus_BitmapGetPixel($hGDIPlusBitmap, 1, 1) ;get pixel color from 1,1 $hImage = _GDIPlus_BitmapCreateFromScan0($iBmpW + (2 * $iBorder), $iBmpH + (2 * $iBorder)) ;create an empty bitmap If @error Then Return SetError(3, 0, False) $hImageCtxt = _GDIPlus_ImageGetGraphicsContext($hImage) ;get the graphics context of the bitmap _GDIPlus_GraphicsSetSmoothingMode($hImageCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY) _GDIPlus_GraphicsClear($hImageCtxt, $iBorderColor) ;clear bitmap with color white _GDIPlus_GraphicsDrawImage($hImageCtxt, $hGDIPlusBitmap, $iBorder, $iBorder) _GDIPlus_ImageDispose($hGDIPlusBitmap) Else $hImage = $hGDIPlusBitmap EndIf $vRet = _GDIPlus_ImageSaveToFile($hImage, $sFileName) ;save bitmap to disk If @error Then Return SetError(4, 0, False) If $vRet Then $vRet = $sFileName ; Cleanup resources _WinAPI_DeleteObject($hBitmap) _WinAPI_DeleteObject($hHBitmap) _GDIPlus_BitmapDispose($hImage) _GDIPlus_BitmapDispose($hGDIPlusBitmap) _GDIPlus_GraphicsDispose($hImageCtxt) _GDIPlus_Shutdown() Return $vRet EndFunc ;==>_ScreenCapture ; #FUNCTION# -------------------------------------------------------------------------------------------------------------------- ; Name...........: _GetText ; Description ...: reading text from a region of the screen ; Syntax.........: _GetText([$iLeft = 0 [, $iTop = 0 [, $iRight = -1 [, $iBottom = -1 [, $iNegative = 0 [, $iBorder = 0 [, $dScale = 1 [, $iBrightness = 0 [, $iContrast = 0 [, $sLanguageTagToUse = Default [, $bUseOcrLine = False]]]]]]]]]]]) ; Parameters ....: $iLeft [optional] X coordinate of the upper left corner of the rectangle ; $iTop [optional] Y coordinate of the upper left corner of the rectangle ; $iRight [optional] X coordinate of the lower right corner of the rectangle. If this is -1, the current screen width will be used ; $iBottom [optional] Y coordinate of the lower right corner of the rectangle. If this is -1, the current screen height will be used. ; $iNegative [optional] 1 = Negative color, 0 = Normal color ; $iBorder [optional] Draw a border araunt, The color is taken from first pixel ; $dScale [optional] Scale factor ; $iBrightness [optional] Integer in the range -255 through 255 that specifies the brightness level. ; $iContrast [optional] Integer in the range -100 through 100 that specifies the contrast level. ; $sLanguageTagToUse [optional] Gets the language being used for text recognition ; $bUseOcrLine [optional] Represents a single line of text recognized by the OCR engine and returned as part of the OcrResult. ; Return value...: Success: Contains the results of Optical Character Recognition (OCR). ; Failure: "" Empty String otherwise. ; On Error: false ; ; Author ........: ; Notes .........: ;-------------------------------------------------------------------------------------------------------------------------------- Func _GetText($iLeft = 0, $iTop = 0, $iRight = -1, $iBottom = -1, $iNegative = 0, $iBorder = 0, $dScale = 1, $iBrightness = 0, $iContrast = 0, $sLanguageTagToUse = Default, $bUseOcrLine = False) Local $hHBitmap, $hBitmap, $hGDIPlusBitmap, $hImage, $hImageCtxt, $sOCRTextResult, $iBmpW, $iBmpH, $iBorderColor _GDIPlus_Startup() $hHBitmap = _ScreenCapture_Capture("", $iLeft, $iTop, $iRight, $iBottom, False) If @error Then Return SetError(1, 0, False) Local $tSIZE = _WinAPI_GetBitmapDimension($hHBitmap) $iBmpW = $dScale * DllStructGetData($tSIZE, 'X') $iBmpH = $dScale * DllStructGetData($tSIZE, 'Y') ;Default ;$iFlags=0,$iIlluminant=0,$iGammaR=10000,$iGammaG=10000,$iGammaB=10000,$iBlack=0,$iWhite=10000,$iContrast=0,$iBrightness=0,$iColorfulness=0,$iTint=0 Local $iIlluminant = 0, $iGammaR = 10000, $iGammaG = 10000, $iGammaB = 10000, $iBlack = 0, $iWhite = 10000, $iColorfulness = 0, $iTint = 0 Local $tAdj = 0 $tAdj = _WinAPI_CreateColorAdjustment($iNegative, $iIlluminant, $iGammaR, $iGammaG, $iGammaB, $iBlack, $iWhite, $iContrast, $iBrightness, $iColorfulness, $iTint) $hBitmap = _WinAPI_AdjustBitmap($hHBitmap, $iBmpW, $iBmpH, $HALFTONE, $tAdj) If @error Then Return SetError(2, 0, False) $hGDIPlusBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBitmap) ; Add Border If $iBorder > 0 Then $iBorderColor = _GDIPlus_BitmapGetPixel($hGDIPlusBitmap, 1, 1) ;get pixel color from 1,1 $hImage = _GDIPlus_BitmapCreateFromScan0($iBmpW + (2 * $iBorder), $iBmpH + (2 * $iBorder)) ;create an empty bitmap If @error Then Return SetError(3, 0, False) $hImageCtxt = _GDIPlus_ImageGetGraphicsContext($hImage) ;get the graphics context of the bitmap _GDIPlus_GraphicsSetSmoothingMode($hImageCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY) _GDIPlus_GraphicsClear($hImageCtxt, $iBorderColor) ;clear bitmap with color white _GDIPlus_GraphicsDrawImage($hImageCtxt, $hGDIPlusBitmap, $iBorder, $iBorder) _GDIPlus_ImageDispose($hGDIPlusBitmap) Else $hImage = $hGDIPlusBitmap EndIf $sOCRTextResult = _UWPOCR_GetText($hImage, $sLanguageTagToUse, $bUseOcrLine) If @error Then Return SetError(4, 0, False) ; Cleanup resources _WinAPI_DeleteObject($hBitmap) _WinAPI_DeleteObject($hHBitmap) _GDIPlus_BitmapDispose($hImage) _GDIPlus_BitmapDispose($hGDIPlusBitmap) _GDIPlus_GraphicsDispose($hImageCtxt) _GDIPlus_Shutdown() Return $sOCRTextResult EndFunc ;==>_GetText ;-------------------------------------------------------------------------------------------------------------------------------- Thank you very much

Thanks so much for this, it adds some much-needed configuration options to UWPOCR and I can confirm it works!

I really don't know much about which one is better only that it gave me better results than tesseract in a project I was doing so that's why I made the wrapper. Saludos

Hi Danyfirex, Good to see you are still developing in AutIT 🙂 I see that this Library is using ONNX Open Neural Network Exchange AI (https://onnx.ai/) Correct ? Can you tell a bit more why we should use this library compared to other conventional OCR libraries ? It is better, faster more reliable, ... PS : I use OCR on a daily basis for different purposes. Thanks

Ufff @Danyfirex, you rock! Thanks! I've been using Tesseract to OCRing this kind of image a lot: A coordinate: [1-255],[1-255] I have trained Tesseract on this kind of image (which I can create with the values I want -> I have the ground truth for training) and it works like a charm! However, it takes too long. With UWPOCR I was able to reduce the time by nearly 90% (yes, there is a "0" after the "9" ). I've read the comment about prepending an image with the characters I want to recognize. I imagine this helps the tool to guess what comes immediately after. I will use the sequence 0-9 and "," but extracted from previous images instead of Arial font. My question is: Is it possible to restrict the set of characters or install a custom language only with those characters?

I've installed Persian and tested it. The GetText throws an error 7 here: _UWPOCR_Log("FAIL __UWPOCR_GetText -> WaitForAsync IOcrResult") So the OCR engine does not seem to respond, that's where it lost me :), sorry, have no further clue. Here's a test sentence:

Hi Malakith, Yes I think anyone with Vista that uses the "$show_capture = 1" parameter will get this error. This code came from another AutoIT forum topic. The code accesses the "Windows Image and Fax Viewer" object that comes with Windows XP. I use this because it supports the TIFF files generated by Tesseract. I believe the "Windows Image and Fax Viewer" has been removed from Windows Vista, and this is why you are getting the error. I use Windows XP myself. If anyone has a TIFF viewer solution for both Win XP and Vista I'd like to know!

Hey.. Im trying to make it work, but i keep getting an error message. When i run the _TesseractCapture it opens the IE window and opens up some gui window, but then it pops up with the message that there is an error in line 221, gives me the gode Line 221 (File "C:\Program Files (x86)"\AutoIt3\Include\Tesseract.au3) $Obj1.ShowFile($capture_filename,1) $Obj1^ERROR Error: Variable must be of type "Object". Ive checked if the paths to the tesseract exe is correct in the tesseract.au3 file and everything looks good, but of some reason its not working. Is there anyone that could explain to me why? Oh and btw, im running Vista Home Premium 64bit. Dunno if that makes a difference? Malakith

Hello AUTOITER, unfortunately it does not provide such feature You can't restrict characters . Saludos