OCR from a small area

[Hashim]

9 minutes ago, ioa747 said:

it is not mine, it belongs to the UWPOCR function
, simply because it passes directly to the function I have added the relevant parameters.
I don't remember I have to look it up

 

Don't worry about it, I was trying to speed up my script to by doing it in memory instead of using  your _ScreenCapture function and relying on the filesystem, but with other optimisations I've managed to make it fast enough for my purposes.

Edited November 5, 2024 by Hashim

[ioa747]

Update the script above to Version: 4.0 

I lightened it up a bit from the legacies it had from the previous script (with big numbers)
which assumed that each Vpoint could be made up of more pixels

so that it is more readable and lighter

[argumentum]

Local $hBitmap = _GDIPlus_BitmapCreateFromFile($sFileName)
    If @error Then Return SetError(1, 0, "") ; because the user ( me ) might do the unexpected  ;)

 

Edited November 6, 2024 by argumentum
English

[ioa747]

I put it, for Version: 5.0

[Hashim]

Hi ioa747, I've run into a problem using the script (both V3 and V4) inside of my For loop over my whole dataset of a million numbers. Everything works fine (bitmap recognition is perfect) until it gets to exactly 9964 (possibly 9965 depending on my maths) in the loop, at which point it crashes with a fatal error (like a MsgBox with no data). This happens no matter what input number I start the loop, so the problem seems to be the 9964th iteration of the loop rather than the number itself. From my debugging it feels like this is something to do with GDIPlus and/or memory consumption, either in the script or in the custom _ScreenCapture function I am using to take the screenshot. Do you have any quick ideas about what might be causing this and how to solve it? If not I can post all of my code tomorrow.

Edited November 6, 2024 by Hashim

[ioa747]

I don't know what to say.
after each call to the custom _ScreenCapture function 
you release the GDIPlus  memory  with

_WinAPI_DeleteObject($hHBitmap)
_GDIPlus_BitmapDispose($hImage)
_GDIPlus_BitmapDispose($hBitmap)
_GDIPlus_GraphicsDispose($hImageCtxt)
_GDIPlus_Shutdown()

like I do in the function _ScreenCapture

 

one more thing
in the numbers you passed us each time there are three numbers in a row
follows that the total will be 999 numbers

there are more in each row?
why need dataset of a million numbers?

Edited November 6, 2024 by ioa747
add _GDIPlus_GraphicsDispose

[ioa747]

You can monitor GDI handle usage in Task Manager:

    Open Task Manager, go to the "Details" tab, right-click on a column header, and select "Select columns."
    Enable "GDI object" to monitor the GDI object count for your script.

[argumentum]

https://www.autoitscript.com/forum/index.php?showtopic=197259&view=findpost&p=1414754 ?

[Nine]

At first glance, you do not use _GDIPlus_GraphicsDispose

[Hashim]

.....

Edited November 11, 2024 by Hashim

[WarMan]

On 2/18/2024 at 3:29 AM, ioa747 said:

first of all i would like to thank @Danyfirex for this wonderful UWPOCR UDF  that he offers us

I noticed that when you perform OCR from a small area of the screen, it doesn't recognize it normally and can't read from it.
and so I proceeded to these functions.
Which work as I expected,
However, I have no experience with GDIPlus.

I post them, to share it with the community, and to get some hint, advice.

In the example below I'm targeting the date on the bottom right of the taskbar,
I have a 1920*1080 screen and 100% scale

expandcollapse popup

; https://www.autoitscript.com/forum/topic/211521-ocr-from-a-small-area/?do=findComment&comment=1530475
#AutoIt3Wrapper_Run_Debug_Mode=Y
#include <GDIPlus.au3>
#include <GUIConstantsEx.au3>
#include <ScreenCapture.au3>
#include <WindowsConstants.au3>
#include "UWPOCR.au3" ; * <-"https://www.autoitscript.com/forum/topic/207324-uwpocr-windows-platform-optical-character-recognition-api-implementation"

_Example()

;--------------------------------------------------------------------------------------------------------------------------------
Func _Example()
    Local $sImageResult, $sOCRTextResult

; 1) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ; From my monitor capture the date of the tskbar = 1808, 1062, 1862, 1074
    $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074)
    ConsoleWrite("$sImageResult=" & $sImageResult & @CRLF)

    ; reading text from $sImageResult
    $sOCRTextResult = _UWPOCR_GetText($sImageResult)
    ConsoleWrite("- 1) OCR Result=" & $sOCRTextResult & @CRLF)

    ShellExecuteWait($sImageResult)

; 2) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ; capture the date and give a border with 20 pixels = 1808, 1062, 1862, 1074, 0, 20
    $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074, 0, 20)

    ; reading text from $sImageResult
    $sOCRTextResult = _UWPOCR_GetText($sImageResult)
    ConsoleWrite("- 2) OCR Result=" & $sOCRTextResult & @CRLF)

    ShellExecuteWait($sImageResult)

; 3) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ; capture the date, give a border with 20 pixels, and invert colors = 1808, 1062, 1862, 1074, 1, 20
    $sImageResult = _ScreenCapture(@ScriptDir & "\tmp_OCR_image.png", 1808, 1062, 1862, 1074, 1, 20)

    ; reading text from $sImageResult
    $sOCRTextResult = _UWPOCR_GetText($sImageResult)
    ConsoleWrite("- 3) OCR Result=" & $sOCRTextResult & @CRLF)

    ShellExecuteWait($sImageResult)

; 4) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    ; and the same from memory

    ConsoleWrite("Delete tmp_OCR_image=" & FileDelete($sImageResult) & @CRLF)

    $sOCRTextResult = _GetText(1808, 1062, 1862, 1074, 1, 20)
    ConsoleWrite("- 4) OCR Result=" & $sOCRTextResult & @CRLF)

EndFunc   ;==>_Example

; #FUNCTION# --------------------------------------------------------------------------------------------------------------------
; Name...........: _ScreenCapture
; Description ...: Captures a region of the screen
; Syntax.........: _ScreenCapture($sFileName [, $iLeft = 0 [, $iTop = 0 [, $iRight = -1 [, $iBottom = -1 [, $iNegative = 0 [, $iBorder = 0 [, $dScale = 1 [, $iBrightness = 0 [, $iContrast = 0]]]]]]]]])
; Parameters ....: $sFileName       Full path and extension of the image file
;                  $iLeft           [optional] X coordinate of the upper left corner of the rectangle
;                  $iTop            [optional] Y coordinate of the upper left corner of the rectangle
;                  $iRight          [optional] X coordinate of the lower right corner of the rectangle. If this is -1, the current screen width will be used
;                  $iBottom         [optional] Y coordinate of the lower right corner of the rectangle. If this is -1, the current screen height will be used.
;                  $iNegative       [optional] 1 = Negative color, 0 = Normal color
;                  $iBorder         [optional] Draw a border araunt, The color is taken from first pixel
;                  $dScale          [optional] Scale factor
;                  $iBrightness     [optional] Integer in the range -255 through 255 that specifies the brightness level.
;                  $iContrast       [optional] Integer in the range -100 through 100 that specifies the contrast level.
; Return value...: Filepath of the image If the image is successfully saved.
;                  False If the image is Not successfully saved.
;
; Author ........:
; Notes .........:
;--------------------------------------------------------------------------------------------------------------------------------
Func _ScreenCapture($sFileName, $iLeft = 0, $iTop = 0, $iRight = -1, $iBottom = -1, $iNegative = 0, $iBorder = 0, $dScale = 1, $iBrightness = 0, $iContrast = 0)
    Local $hHBitmap, $hBitmap, $hGDIPlusBitmap, $hImage, $hImageCtxt, $vRet, $iBmpW, $iBmpH, $iBorderColor
    _GDIPlus_Startup()

    $hHBitmap = _ScreenCapture_Capture("", $iLeft, $iTop, $iRight, $iBottom, False)
    If @error Then Return SetError(1, 0, False)

    Local $tSIZE = _WinAPI_GetBitmapDimension($hHBitmap)
    $iBmpW = $dScale * DllStructGetData($tSIZE, 'X')
    $iBmpH = $dScale * DllStructGetData($tSIZE, 'Y')

    ;Default
    ;$iFlags=0,$iIlluminant=0,$iGammaR=10000,$iGammaG=10000,$iGammaB=10000,$iBlack=0,$iWhite=10000,$iContrast=0,$iBrightness=0,$iColorfulness=0,$iTint=0
    Local $iIlluminant = 0, $iGammaR = 10000, $iGammaG = 10000, $iGammaB = 10000, $iBlack = 0, $iWhite = 10000, $iColorfulness = 0, $iTint = 0
    Local $tAdj = 0
    $tAdj = _WinAPI_CreateColorAdjustment($iNegative, $iIlluminant, $iGammaR, $iGammaG, $iGammaB, $iBlack, $iWhite, $iContrast, $iBrightness, $iColorfulness, $iTint)

    $hBitmap = _WinAPI_AdjustBitmap($hHBitmap, $iBmpW, $iBmpH, $HALFTONE, $tAdj)
    If @error Then Return SetError(2, 0, False)
    $hGDIPlusBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBitmap)

    ; Add Border
    If $iBorder > 0 Then
        $iBorderColor = _GDIPlus_BitmapGetPixel($hGDIPlusBitmap, 1, 1) ;get pixel color from 1,1
        $hImage = _GDIPlus_BitmapCreateFromScan0($iBmpW + (2 * $iBorder), $iBmpH + (2 * $iBorder)) ;create an empty bitmap
        If @error Then Return SetError(3, 0, False)

        $hImageCtxt = _GDIPlus_ImageGetGraphicsContext($hImage) ;get the graphics context of the bitmap
        _GDIPlus_GraphicsSetSmoothingMode($hImageCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY)
        _GDIPlus_GraphicsClear($hImageCtxt, $iBorderColor) ;clear bitmap with color white
        _GDIPlus_GraphicsDrawImage($hImageCtxt, $hGDIPlusBitmap, $iBorder, $iBorder)

        _GDIPlus_ImageDispose($hGDIPlusBitmap)
    Else
        $hImage = $hGDIPlusBitmap
    EndIf

    $vRet = _GDIPlus_ImageSaveToFile($hImage, $sFileName)  ;save bitmap to disk
    If @error Then Return SetError(4, 0, False)
    If $vRet Then $vRet = $sFileName

    ; Cleanup resources
    _WinAPI_DeleteObject($hBitmap)
    _WinAPI_DeleteObject($hHBitmap)
    _GDIPlus_BitmapDispose($hImage)
    _GDIPlus_BitmapDispose($hGDIPlusBitmap)
    _GDIPlus_GraphicsDispose($hImageCtxt)
    _GDIPlus_Shutdown()

    Return $vRet

EndFunc   ;==>_ScreenCapture

; #FUNCTION# --------------------------------------------------------------------------------------------------------------------
; Name...........: _GetText
; Description ...: reading text from a region of the screen
; Syntax.........: _GetText([$iLeft = 0 [, $iTop = 0 [, $iRight = -1 [, $iBottom = -1 [, $iNegative = 0 [, $iBorder = 0 [, $dScale = 1 [, $iBrightness = 0 [, $iContrast = 0 [, $sLanguageTagToUse = Default [, $bUseOcrLine = False]]]]]]]]]]])
; Parameters ....: $iLeft           [optional] X coordinate of the upper left corner of the rectangle
;                  $iTop            [optional] Y coordinate of the upper left corner of the rectangle
;                  $iRight          [optional] X coordinate of the lower right corner of the rectangle. If this is -1, the current screen width will be used
;                  $iBottom         [optional] Y coordinate of the lower right corner of the rectangle. If this is -1, the current screen height will be used.
;                  $iNegative       [optional] 1 = Negative color, 0 = Normal color
;                  $iBorder         [optional] Draw a border araunt, The color is taken from first pixel
;                  $dScale          [optional] Scale factor
;                  $iBrightness     [optional] Integer in the range -255 through 255 that specifies the brightness level.
;                  $iContrast       [optional] Integer in the range -100 through 100 that specifies the contrast level.
;                  $sLanguageTagToUse   [optional] Gets the language being used for text recognition
;                  $bUseOcrLine         [optional] Represents a single line of text recognized by the OCR engine and returned as part of the OcrResult.
; Return value...: Success: Contains the results of Optical Character Recognition (OCR).
;                  Failure: "" Empty String otherwise.
;                  On Error: false
;
; Author ........:
; Notes .........:
;--------------------------------------------------------------------------------------------------------------------------------
Func _GetText($iLeft = 0, $iTop = 0, $iRight = -1, $iBottom = -1, $iNegative = 0, $iBorder = 0, $dScale = 1, $iBrightness = 0, $iContrast = 0, $sLanguageTagToUse = Default, $bUseOcrLine = False)
    Local $hHBitmap, $hBitmap, $hGDIPlusBitmap, $hImage, $hImageCtxt, $sOCRTextResult, $iBmpW, $iBmpH, $iBorderColor
    _GDIPlus_Startup()

    $hHBitmap = _ScreenCapture_Capture("", $iLeft, $iTop, $iRight, $iBottom, False)
    If @error Then Return SetError(1, 0, False)

    Local $tSIZE = _WinAPI_GetBitmapDimension($hHBitmap)
    $iBmpW = $dScale * DllStructGetData($tSIZE, 'X')
    $iBmpH = $dScale * DllStructGetData($tSIZE, 'Y')

    ;Default
    ;$iFlags=0,$iIlluminant=0,$iGammaR=10000,$iGammaG=10000,$iGammaB=10000,$iBlack=0,$iWhite=10000,$iContrast=0,$iBrightness=0,$iColorfulness=0,$iTint=0
    Local $iIlluminant = 0, $iGammaR = 10000, $iGammaG = 10000, $iGammaB = 10000, $iBlack = 0, $iWhite = 10000, $iColorfulness = 0, $iTint = 0
    Local $tAdj = 0
    $tAdj = _WinAPI_CreateColorAdjustment($iNegative, $iIlluminant, $iGammaR, $iGammaG, $iGammaB, $iBlack, $iWhite, $iContrast, $iBrightness, $iColorfulness, $iTint)

    $hBitmap = _WinAPI_AdjustBitmap($hHBitmap, $iBmpW, $iBmpH, $HALFTONE, $tAdj)
    If @error Then Return SetError(2, 0, False)
    $hGDIPlusBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBitmap)

    ; Add Border
    If $iBorder > 0 Then
        $iBorderColor = _GDIPlus_BitmapGetPixel($hGDIPlusBitmap, 1, 1) ;get pixel color from 1,1
        $hImage = _GDIPlus_BitmapCreateFromScan0($iBmpW + (2 * $iBorder), $iBmpH + (2 * $iBorder)) ;create an empty bitmap
        If @error Then Return SetError(3, 0, False)

        $hImageCtxt = _GDIPlus_ImageGetGraphicsContext($hImage) ;get the graphics context of the bitmap
        _GDIPlus_GraphicsSetSmoothingMode($hImageCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY)
        _GDIPlus_GraphicsClear($hImageCtxt, $iBorderColor) ;clear bitmap with color white
        _GDIPlus_GraphicsDrawImage($hImageCtxt, $hGDIPlusBitmap, $iBorder, $iBorder)

        _GDIPlus_ImageDispose($hGDIPlusBitmap)
    Else
        $hImage = $hGDIPlusBitmap
    EndIf

    $sOCRTextResult = _UWPOCR_GetText($hImage, $sLanguageTagToUse, $bUseOcrLine)
    If @error Then Return SetError(4, 0, False)

    ; Cleanup resources
    _WinAPI_DeleteObject($hBitmap)
    _WinAPI_DeleteObject($hHBitmap)
    _GDIPlus_BitmapDispose($hImage)
    _GDIPlus_BitmapDispose($hGDIPlusBitmap)
    _GDIPlus_GraphicsDispose($hImageCtxt)
    _GDIPlus_Shutdown()

    Return $sOCRTextResult

EndFunc   ;==>_GetText
;--------------------------------------------------------------------------------------------------------------------------------

Thank you very much 

in _GetText() You used Return 4 times before cleaning resources
 

expandcollapse popup

Func _GetText($iLeft = 0, $iTop = 0, $iRight = -1, $iBottom = -1, $iNegative = 0, $iBorder = 0, $dScale = 1, $iBrightness = 0, $iContrast = 0, $sLanguageTagToUse = Default, $bUseOcrLine = False)
    Local $hHBitmap, $hBitmap, $hGDIPlusBitmap, $hImage, $hImageCtxt, $sOCRTextResult, $iBmpW, $iBmpH, $iBorderColor
    _GDIPlus_Startup()

    $hHBitmap = _ScreenCapture_Capture("", $iLeft, $iTop, $iRight, $iBottom, False)
    If @error Then
        _GDIPlus_Shutdown()
        Return SetError(1, 0, False)
    EndIf

    Local $tSIZE = _WinAPI_GetBitmapDimension($hHBitmap)
    $iBmpW = $dScale * DllStructGetData($tSIZE, 'X')
    $iBmpH = $dScale * DllStructGetData($tSIZE, 'Y')

    ;Default
    ;$iFlags=0,$iIlluminant=0,$iGammaR=10000,$iGammaG=10000,$iGammaB=10000,$iBlack=0,$iWhite=10000,$iContrast=0,$iBrightness=0,$iColorfulness=0,$iTint=0
    Local $iIlluminant = 0, $iGammaR = 10000, $iGammaG = 10000, $iGammaB = 10000, $iBlack = 0, $iWhite = 10000, $iColorfulness = 0, $iTint = 0
    Local $tAdj = 0
    $tAdj = _WinAPI_CreateColorAdjustment($iNegative, $iIlluminant, $iGammaR, $iGammaG, $iGammaB, $iBlack, $iWhite, $iContrast, $iBrightness, $iColorfulness, $iTint)

    $hBitmap = _WinAPI_AdjustBitmap($hHBitmap, $iBmpW, $iBmpH, $HALFTONE, $tAdj)
    If @error Then
        _GDIPlus_Shutdown()
        Return SetError(2, 0, False)
    EndIf
    $hGDIPlusBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBitmap)

    ; Add Border
    If $iBorder > 0 Then
        $iBorderColor = _GDIPlus_BitmapGetPixel($hGDIPlusBitmap, 1, 1) ;get pixel color from 1,1
        $hImage = _GDIPlus_BitmapCreateFromScan0($iBmpW + (2 * $iBorder), $iBmpH + (2 * $iBorder)) ;create an empty bitmap
        If @error Then
            _GDIPlus_BitmapDispose($hGDIPlusBitmap)
            _GDIPlus_Shutdown()
            Return SetError(3, 0, False)
        EndIf

        $hImageCtxt = _GDIPlus_ImageGetGraphicsContext($hImage) ;get the graphics context of the bitmap
        _GDIPlus_GraphicsSetSmoothingMode($hImageCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY)
        _GDIPlus_GraphicsClear($hImageCtxt, $iBorderColor) ;clear bitmap with color white
        _GDIPlus_GraphicsDrawImage($hImageCtxt, $hGDIPlusBitmap, $iBorder, $iBorder)

        _GDIPlus_ImageDispose($hGDIPlusBitmap)
    Else
        $hImage = $hGDIPlusBitmap
    EndIf

    $sOCRTextResult = _UWPOCR_GetText($hImage, $sLanguageTagToUse, $bUseOcrLine)
    Local $iError = @error

    ; Cleanup resources
    _WinAPI_DeleteObject($hBitmap)
    _WinAPI_DeleteObject($hHBitmap)
    _GDIPlus_BitmapDispose($hImage)
    _GDIPlus_BitmapDispose($hGDIPlusBitmap)
    _GDIPlus_GraphicsDispose($hImageCtxt)
    _GDIPlus_Shutdown()

    Return SetError($iError = 0 ? 0 : 4, 0, $iError = 0 ? $sOCRTextResult : False)

EndFunc   ;==>_GetText

 

Edited February 1 by WarMan

[ioa747]

@WarMan

based on your observation, revised the code in the original 1st post
thank you very much