Jump to content

Another bitmap brute-forcing thread

Hashim
 Share

Recommended Posts

Hashim Seeker

Hashim

Posted
Posted (edited)

Happy New Year everyone, this is a successor thread of sorts to one from a few months ago, I've now tested @Werty's latest solution from that thread:
  

;Be sure $posx and $posy is pointing at the correct spot!
Global $posx = 1718, $posy = 699, $result = "", $code[3]
;Lookup table to avoid searching
Global $digit[115] = [1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,6,0,0,0,3,2,0,0,0,0, _
                      0,0,0,0,0,0,0,0,9,0,0,0,0,0,8,0,0,0,0,0, _
                      0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,5]

;Instead of screencapture_capture()
    $hDDC = _WinAPI_GetDC(0)
    $hCDC = _WinAPI_CreateCompatibleDC($hDDC)
    $hBmp = _WinAPI_CreateCompatibleBitmap($hDDC, 34, 20)
            _WinAPI_SelectObject($hCDC, $hBmp)
            _WinAPI_BitBlt($hCDC, 0, 0, 34, 20, $hDDC, $posx, $posy, 0x00CC0020)
            _WinAPI_DeleteDC($hCDC)

    $result = Getnumber($result)

Func Getnumber($result)
   
    Local $code[3] = [0,0,0], $value = 64, $pixels = DllStructCreate('dword[680]')
    DllCall('gdi32.dll', 'dword', 'GetBitmapBits', 'ptr', $hBmp, 'dword', DllStructGetSize($pixels), 'ptr', DllStructGetPtr($pixels))

For $loop = 1 To 680 Step 102
    $code[0] += DllStructGetData($pixels, 1, $loop   ) = 4278255360 ? $value:0
    $code[1] += DllStructGetData($pixels, 1, $loop+16) = 4278255360 ? $value:0
    $code[2] += DllStructGetData($pixels, 1, $loop+32) = 4278255360 ? $value:0
    $value /= 2
Next
    Return String($digit[$code[0]]) & String($digit[$code[1]]) & String($digit[$code[2]])
EndFunc

 

I can confirm it works as well for that tool as his original lockbits solution, and is beautifully concise compared to it. I don't have the time to test on the whole dataset so can't confirm how much faster it is but on the few thousand results I did check it definitely doesn't seem slower or less accurate, so I'll take his word for it that this is now the best solution. The problem now is that I need to do the same thing for a similar but newer internal tool, which seems to have been built by the same dev but uses a completely different bitmap font for its results, and the results are also 4 digits instead of 3. It also seems to be in "italics" so I'm not sure whether this makes the solution impossible or if the bitmaps are still uniform enough for the maths to work. The only really good news is that the font is larger than in the original tool.

Would it possible to adapt the maths of this solution to these new bitmaps? If I understood how it worked I'd do it myself, but it's way over my head and so was @AndyG's explanation of it, but I have provided all of the possible digits below and can provide more examples if needed.

1.png

2.png

3.png

4.png

Edited by Hashim
Hashim Seeker

Hashim

Posted
  • Author
Posted (edited)

Anyone at all? Here's the full script I'm using along with my best-guess attempts to adapt the results and constants for a 4-digit result - like changing $value from 64 to 128 and adding loop+64. The program seems to run fine without errors, but the digits are all being recognised as either 1s or 0s (the first two digits always as 1s).

I'm happy to buy a drink or five for anyone that can figure this out since management have realised I'm the only person in the department that can extract the data from these old tools. 

#include <WinAPI.au3>

Run("tool.exe")
Local $tool = WinWaitActive("tool", "", 3)

$db = FileOpen(@ScriptDir & "\serials.csv", 1)

;Be sure $posx and $posy is pointing at the correct spot!
Global $posx = 829, $posy = 456, $result = "", $code[4]
;Lookup table to avoid searching
Global $digit[115] = [1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,6,0,0,0,3,2,0,0,0,0, _
                      0,0,0,0,0,0,0,0,9,0,0,0,0,0,8,0,0,0,0,0, _
                      0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,5]

For $loop = 0 To 999999

    Do
        $serial = StringFormat('%06i', $loop)
        ControlClick($tool, "", "TTabSheet1", "primary", 1, 100, 100)
        $hDDC = _WinAPI_GetDC(0)
        $hCDC = _WinAPI_CreateCompatibleDC($hDDC)
        $hBmp = _WinAPI_CreateCompatibleBitmap($hDDC, 34, 20)
        _WinAPI_SelectObject($hCDC, $hBmp)
        _WinAPI_BitBlt($hCDC, 0, 0, 34, 20, $hDDC, $posx, $posy, 0x00CC0020)
        _WinAPI_DeleteDC($hCDC)
        $result = GetNumber($result)
    Until ((StringLen($result) = 4) and StringIsDigit($result))

    FileWrite($db, $serial & "," & $result & @LF)

Next

FileClose($db)
Exit

Func GetNumber($result)

    Local $code[4] = [0,0,0,0], $value = 128, $pixels = DllStructCreate('dword[680]')
    DllCall('gdi32.dll', 'dword', 'GetBitmapBits', 'ptr', $hBmp, 'dword', DllStructGetSize($pixels), 'ptr', DllStructGetPtr($pixels))

For $loop = 1 To 680 Step 102
    $code[0] += DllStructGetData($pixels, 1, $loop   ) = 4278255360 ? $value:0
    $code[1] += DllStructGetData($pixels, 1, $loop+16) = 4278255360 ? $value:0
    $code[2] += DllStructGetData($pixels, 1, $loop+32) = 4278255360 ? $value:0
    $code[3] += DllStructGetData($pixels, 1, $loop+64) = 4278255360 ? $value:0
    $value /= 2
Next
    Return String($digit[$code[0]]) & String($digit[$code[1]]) & String($digit[$code[2]]) & String($digit[$code[3]])
EndFunc

 

Edited by Hashim
junkew Universalist

junkew

Posted
Posted (edited)

 

Maybe this helps a little in understanding. 
Its just manipulation of pixels in a certan layout (PNG). 
Look at the output and zoom in/out, you can see almost the numbers yourself  

#Region includes
Opt('MustDeclareVars', 1)
;#include <GUIConstants.au3>
#include <GDIPlus.au3>
#Include <ScreenCapture.au3>
#include <string.au3>
#include <WinAPI.au3>
#include <WindowsConstants.au3>
#EndRegion

_GDIPlus_Startup()

Dim $BitmapFilename
Dim $BMPData="", $BMPWidth=0, $BMPHeight=0, $BMPLineWidth=0, $imgBytes=1
dim $binaryLine

for $i=1 to 4
    $BitmapFilename = @TempDir & "\" & "pic" & string($i) & ".png"
        $BMPData=""
        $BMPWidth=0
        $BMPHeight=0
        $BMPLineWidth=0
        $imgBytes=1
        consolewrite($bitmapFileName)

; Load the bitmap to find
    getImage($BitmapFilename, $BMPData, $BMPWidth, $BMPHeight, $BMPLineWidth, $imgBytes)

    ;~ See the height and width and dump the data on screen
    consolewrite($BMPHeight & " ; ")
    consolewrite($BMPWidth &@CRLF )
    consolewrite("0         1         2         3         4         5         6         7         8         9         x         x         x         x         " & @CRLF)
    consolewrite("01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" & @CRLF)
    For $iLine = 11 To $BMPHeight-12

    ;~  From the screenshots we can ignore the first 16 bytes (remember in text its 2 characters
    ;~  $binaryLine=BinaryMid($bmp1Data,($i*$BMP1LineWidth)+16,$BMP1LineWidth)
    ;~  and we only need 4 digits which takes about 28 characters and 1 character separator =
        $binaryLine=BinaryMid($bmpData,($iLine *$BMPLineWidth)+16,60)
;~ Lets make it readable
        $binaryLine=StringReplace($binaryLine,"0",".")

        consolewrite($iLine & " : " & $binaryLine & @CRLF)
    next

;~  lets recognise
    for $iNumber=1 to 4

        consolewrite(@crlf)
            consolewrite("0         1         2         3         4         5         6         7         8         9         x         x         x         x         " & @CRLF)
    consolewrite("01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" & @CRLF)

        For $iLine = 11 To $BMPHeight-12
            $binaryLine=BinaryMid($bmpData,($iLine *$BMPLineWidth)+16+(($iNumber-1)*15),14)
            $binaryLine=StringReplace($binaryLine,"0",".")
            consolewrite($iLine & " : " & $binaryLine & @CRLF)
        next
    Next

Next

_GDIPlus_Shutdown()

Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3)
local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle, $bitMapdata, $pixelFormat

; Load the bitmap to search in
    If $BMPFile="SCREEN" Then
        $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
;try to get a handle
        $handle = WinGetHandle($BMPFile)
        If @error Then
;Assume its an unknown handle so correct filename should be given
            $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)
;~          $hBitmap = _GDIPlus_ImageLoadFromFile($BMPFile)
        Else
            $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False)
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
        EndIf
    EndIf

;Get $tagGDIPBITMAPDATA structure
  ConsoleWrite("Bitmap Width:    " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison
    if ($imgBytes=1) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD,  $GDIP_PXF04INDEXED)
    Endif
    if ($imgBytes=2) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    EndIf
    if ($imgBytes=3) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    endIf

    If @ERROR Then MsgBox(0,"","Error locking region " & @error)

    $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap.
    $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap.

    $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData")

    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose ($pBitmap)
    _WinAPI_DeleteObject ($pBitmap)

EndFunc;==>GetImage
; Draw rectangle on screen.
Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
    Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2
    $x1 = $tLeft
    $x2 = $tRight
    $y1 = $tTop
    $y2 = $tBottom
    $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop)
    $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color)
    $obj_orig = _WinAPI_SelectObject($hDC, $hPen)

    _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right
    _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right
    _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right
    _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left

    ; clear resources
    _WinAPI_SelectObject($hDC, $obj_orig)
    _WinAPI_DeleteObject($hPen)
    _WinAPI_ReleaseDC(0, $hDC)
EndFunc   ;==>_UIA_DrawRect
#EndRegion



I assume you refer to 

 

Edited by junkew
updated script to have 8 bits per pixel for easier seeing

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

junkew Universalist

junkew

Posted
Posted

You should get some output like below, copy pasted to a texteditor (replaced 0 with a dot) you can find out the width of the numbers somewhere around 28 (depending on your colordepth)
  

.x................................................................................................................................................................................
.x....................................................BBB.....................BBB..BBB................BBB..BBB..BBB.....................BBB.......................................
.x...................................................B222B...................B222BB222B..............B222BB222BB222B...................B222B......................................
.x...................................................B222B...................B222BB222B..............B222BB222BB222B...................B222B......................................
.x....................................................BBB.....................BBB..BBB................BBB..BBB..BBB.....................BBB.......................................
.x................................................................................................................................................................................
.x..............................................BBB..BBB................BBB.....................BBB.................BBB...........BBB..BBB........................................
.x.............................................B222BB222B..............B222B...................B222B...............B222B.........B222BB222B.......................................
.x.............................................B222BB222B..............B222B...................B222B...............B222B.........B222BB222B.......................................
.x..............................................BBB..BBB................BBB.....................BBB.................BBB...........BBB..BBB........................................
.x................................................................................................................................................................................
.x.........................................BBB.......BBB...........BBB..........................BBB.................BBB................BBB........................................
.x........................................B222B.....B222B.........B222B........................B222B...............B222B..............B222B.......................................
.x........................................B222B.....B222B.........B222B........................B222B...............B222B..............B222B.......................................
.x.........................................BBB.......BBB...........BBB..........................BBB.................BBB................BBB........................................
.x................................................................................................................................................................................
.x...................................BBB............BBB...........BBB..BBB..BBB..BBB................BBB..BBB..BBB..BBB................BBB.........................................
.x..................................B222B..........B222B.........B222BB222BB222BB222B..............B222BB222BB222BB222B..............B222B........................................
.x..................................B222B..........B222B.........B222BB222BB222BB222B..............B222BB222BB222BB222B..............B222B........................................
.x...................................BBB............BBB...........BBB..BBB..BBB..BBB................BBB..BBB..BBB..BBB................BBB.........................................
.x................................................................................................................................................................................
.x..................................BBB..BBB..BBB..BBB..BBB......BBB.................BBB..........................BBB................BBB..........................................
.x.................................B222BB222BB222BB222BB222B....B222B...............B222B........................B222B..............B222B.........................................
.x.................................B222BB222BB222BB222BB222B....B222B...............B222B........................B222B..............B222B.........................................
.x..................................BBB..BBB..BBB..BBB..BBB......BBB.................BBB..........................BBB................BBB..........................................
.x................................................................................................................................................................................
.x................................................BBB...........BBB.................BBB.....................BBB.....................BBB...........................................
.x...............................................B222B.........B222B...............B222B...................B222B...................B222B..........................................
.x...............................................B222B.........B222B...............B222B...................B222B...................B222B..........................................
.x................................................BBB...........BBB.................BBB.....................BBB.....................BBB...........................................
.x................................................................................................................................................................................
.x................................................BBB................BBB..BBB..BBB................BBB..BBB.....................BBB..BBB..BBB......................................
.x...............................................B222B..............B222BB222BB222B..............B222BB222B...................B222BB222BB222B.....................................
.x...............................................B222B..............B222BB222BB222B..............B222BB222B...................B222BB222BB222B.....................................
.x................................................BBB................BBB..BBB..BBB................BBB..BBB.....................BBB..BBB..BBB......................................

 

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

junkew Universalist

junkew

Posted
Posted

I enhanced the script of my first answer.
Based on that you can see the patterns and identify the differences between the numbers 0-9

It depends if your are "reading" bitblt from the screen you should basically iterate per character.

If you save them all as png files its like the given script needs to be enhanced and add the pattern later onwards.
 

 

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

junkew Universalist

junkew

Posted
Posted (edited)

So here some partial recognition based on the png files. As its italic it will be more complicated to properly first cut the characters nicely.

I hope the script gives you some insights on how to do the analysis.

When thats done you could directly bitblit line11 and 27 and 41 from the screen as that seems to be a proper discriminator to identify the number.
From the screen you can probably directly bitblt with black/white pixels directly which should speed up even more.

Due to italic it seems a little harder to calculate the exact position for each digit calculated (maybe even easier to just hardcode them as it seems to be off by 1 or 2 bytes for each digit. 

#Region includes
Opt('MustDeclareVars', 1)
;#include <GUIConstants.au3>
#include <GDIPlus.au3>
#Include <ScreenCapture.au3>
#include <string.au3>
#include <WinAPI.au3>
#include <WindowsConstants.au3>
#EndRegion

_GDIPlus_Startup()

Dim $BitmapFilename
Dim $BMPData="", $BMPWidth=0, $BMPHeight=0, $BMPLineWidth=0, $imgBytes=2
dim $binaryLine, $startColumn

;~ for each picture file
for $i=1 to 4
    $BitmapFilename = @TempDir & "\" & "pic" & string($i) & ".png"
        $BMPData=""
        $BMPWidth=0
        $BMPHeight=0
        $BMPLineWidth=0
        $imgBytes=1
        consolewrite($bitmapFileName)

; Load the bitmap to find
    getImage($BitmapFilename, $BMPData, $BMPWidth, $BMPHeight, $BMPLineWidth, $imgBytes)

    ;~ See the height and width and dump the data on screen
    consolewrite($BMPHeight & " ; ")
    consolewrite($BMPWidth &@CRLF )
    consolewrite("       0         1         2         3         4         5         6         7         8         9         x         x         x         x         " & @CRLF)
    consolewrite("       01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" & @CRLF)
    For $iLine = 11 To $BMPHeight-12

    ;~  From the screenshots we can ignore the first 16 bytes (remember in text its 2 characters
    ;~  $binaryLine=BinaryMid($bmp1Data,($i*$BMP1LineWidth)+16,$BMP1LineWidth)
    ;~  and we only need 4 digits which takes about 28 characters and 1 character separator =
        $binaryLine=BinaryMid($bmpData,($iLine *$BMPLineWidth)+16,60)
;~      $binaryLine=$bmpData
;~ Lets make it readable
        $binaryLine=StringReplace($binaryLine,"0",".")

        consolewrite($iLine & " : " & $binaryLine & @CRLF)
    next

;~  lets recognise
    for $iNumber=1 to 4

        consolewrite(@crlf)
            consolewrite("0         1         2         3         4         5         6         7         8         9         x         x         x         x         " & @CRLF)
    consolewrite("01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" & @CRLF)
    
;~  Its hard to determine the column for each digit in a formula 
        if $iNumber=1 then $startColumn=16
        if $iNumber=2 then $startColumn=30
        if $iNumber=3 then $startColumn=45
        if $iNumber=4 then $startColumn=59

;~ Only a few lines in the png are of interest

        For $iLine = 11 To $BMPHeight-12
;~          $binaryLine=BinaryMid($bmpData,($iLine *$BMPLineWidth)+16+(($iNumber-1)*14),15)
            $binaryLine=BinaryMid($bmpData,($iLine *$BMPLineWidth)+$startColumn,15)


            $binaryLine=StringReplace($binaryLine,"0"," ")
            consolewrite($iLine & " : " & $binaryLine & @CRLF)
        next
;~      if on line 11 at position 28 I have a B it can be a 4

;~ It seems to be easier to revert back to a string when we have the data
        local $lineData11=stringstripws(StringRegExpReplace(binaryMid($bmpData,(11*$BMPLineWidth)+$startColumn,15),"[x0]"," "),3)
        local $lineData27=stringstripws(StringRegExpReplace(binaryMid($bmpData,(27*$BMPLineWidth)+$startColumn,15),"[x0]"," "),3)
        local $lineData41=stringstripws(StringRegExpReplace(binaryMid($bmpData,(41*$BMPLineWidth)+$startColumn,15),"[x0]"," "),3)

        consolewrite(string($lineData11) & " - ")
        consolewrite(string($lineData27) & " - ")
        consolewrite(string($lineData41) & @crlf)

;~ And here determine the magic lines

        if (($lineData11="BBB")                     and ($lineData27="B222B          B222B")) then consolewrite("Ita a 4" & @CRLF)
        if (($lineData11="BBB  BBB")                and ($lineData27="B222BB222BB222BB222B")) then consolewrite("Ita a 6" & @CRLF)
        if (($lineData11="BBB  BBB  BBB")           and ($lineData27="B222BB222BB222BB222B")) then consolewrite("Ita a 9" & @CRLF)
        if (($lineData11="BBB")                     and ($lineData27="B222B")) then consolewrite("Ita a 1" & @CRLF)

        if (($lineData11="BBB  BBB  BBB")           and ($lineData27="B222BB222BB222B")) then consolewrite("Ita a 8" & @CRLF)
        if (($lineData11="BBB  BBB  BBB")           and ($lineData27="B222B     B222B     B222B")) then consolewrite("Ita a 0" & @CRLF)

        if (($lineData11="BBB  BBB  BBB")           and ($lineData27="B222BB222B") and ($lineData41="BBB  BBB  BBB")) then consolewrite("Ita a 3" & @CRLF)
        if (($lineData11="BBB  BBB  BBB")           and ($lineData27="B222BB222B") and ($lineData41="BBB  BBB  BBB  BBB  BBB")) then consolewrite("Ita a 2" & @CRLF)

        if (($lineData11="BBB  BBB  BBB  BBB  BBB")           and ($lineData27="B222B") and ($lineData41="BBB")) then consolewrite("Ita a 7" & @CRLF)
        if (($lineData11="BBB  BBB  BBB  BBB  BBB")           and ($lineData27="B222B") and ($lineData41="BBB  BBB  BBB")) then consolewrite("Ita a 5" & @CRLF)
    Next

Next

_GDIPlus_Shutdown()

Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3)
local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle, $bitMapdata, $pixelFormat

; Load the bitmap to search in
    If $BMPFile="SCREEN" Then
        $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
;try to get a handle
        $handle = WinGetHandle($BMPFile)
        If @error Then
;Assume its an unknown handle so correct filename should be given
            $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)
;~          $hBitmap = _GDIPlus_ImageLoadFromFile($BMPFile)
        Else
            $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False)
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
        EndIf
    EndIf

;Get $tagGDIPBITMAPDATA structure
  ConsoleWrite("Bitmap Width:    " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison
    if ($imgBytes=1) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD,    $GDIP_PXF04INDEXED)
    Endif
    if ($imgBytes=2) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    EndIf
    if ($imgBytes=3) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    endIf

    If @ERROR Then MsgBox(0,"","Error locking region " & @error)

    $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap.
    $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap.

    $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData")

    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose ($pBitmap)
    _WinAPI_DeleteObject ($pBitmap)

EndFunc;==>GetImage
; Draw rectangle on screen.
Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
    Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2
    $x1 = $tLeft
    $x2 = $tRight
    $y1 = $tTop
    $y2 = $tBottom
    $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop)
    $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color)
    $obj_orig = _WinAPI_SelectObject($hDC, $hPen)

    _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right
    _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right
    _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right
    _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left

    ; clear resources
    _WinAPI_SelectObject($hDC, $obj_orig)
    _WinAPI_DeleteObject($hPen)
    _WinAPI_ReleaseDC(0, $hDC)
EndFunc   ;==>_UIA_DrawRect
#EndRegion

 

Edited by junkew

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

Werty Universalist

Werty

Posted
Posted
On 1/22/2025 at 3:14 AM, Hashim said:

and can provide more examples if needed

Some examples where the first digit is 1, preferable 10 images where all 0 through 9 is the first digit, now that they are cursive.

The loop probably wont be as simple as in the old thread, but need the images to be sure.

Some guy's script + some other guy's script = my script!

Nine Universalist

Nine

Posted
Posted (edited)

Here another way based on relative pixel location from the first (left/top most) pixel using my ScreenScraping UDF.  Coordinates are screen absolute location. 

#include "C:\Apps\AutoIt\GetScreen\GetScreen.au3"
#include <Color.au3>

Local $hWnd = WinActivate("Microsoft Office Picture Manager")
WinWaitActive($hWnd)

Global Const $LINE = 423, $COLUMN = 550, $ENDX = 690, $ENDY = 465, $NDIGIT = 4
Global $aDigit = DefineNum()
_GetScreen_Initialize($ENDX, $ENDY)

Local $hTimer = TimerInit()
_GetScreen_GetScreen()

Local $iPos = $COLUMN, $vValue
For $i = 1 To $NDIGIT
  $vValue &= GetDigit($iPos)
  $iPos = @extended
Next
ConsoleWrite($vValue & @CRLF)
ConsoleWrite(TimerDiff($hTimer) & @CRLF)

Func GetDigit($iStart)
  Local $aTmp
  For $i = $iStart To $ENDX
    If IsColor(_GetScreen_GetPixel($i, $LINE)) Then
      For $j = 0 To UBound($aDigit) - 1
        $aTmp = $aDigit[$j][2]
        For $k = 0 To UBound($aTmp) - 1
          If Not IsColor(_GetScreen_GetPixel($i + $aTmp[$k][0], $LINE + $aTmp[$k][1])) Then ContinueLoop 2
        Next
        Return SetExtended($i + $aDigit[$j][1], $aDigit[$j][0])
      Next
      ConsoleWrite("error" & @CRLF)
      Return SetError(1)
    EndIf
  Next
EndFunc   ;==>GetDigit

Func DefineNum()
  Local $aNum[10][3] = [[0, 14, 0], [1, 4, 0], [2, 14, 0], [8, 14, 0], [3, 14, 0], [5, 24, 0], [7, 24, 0], [4, 4, 0], [9, 24, 0], [6, 10, 0]]
  Local $aTmp = [[13, 15], [-3, 20], [9, 10]]         ; 0
  $aNum[0][2] = $aTmp
  Local $aTmp = [[-1, 10], [-2, 15], [-9, 30]]        ; 1
  $aNum[1][2] = $aTmp
  Local $aTmp = [[-3, 20], [-9, 30], [11, 30]]        ; 2
  $aNum[2][2] = $aTmp
  Local $aTmp = [[-4, 10], [-8, 20], [-2, 15]]        ; 8
  $aNum[3][2] = $aTmp
  Local $aTmp = [[8, 15], [3, 15], [-6, 5], [11, 25]] ; 3
  $aNum[4][2] = $aTmp
  Local $aTmp = [[20, 0], [4, 10], [9, 10]]           ; 5
  $aNum[5][2] = $aTmp
  Local $aTmp = [[20, 0], [19, 5], [2, 20]]           ; 7
  $aNum[6][2] = $aTmp
  Local $aTmp = [[2, 20], [-18, 20], [-4, 25]]        ; 4
  $aNum[7][2] = $aTmp
  Local $aTmp = [[-6, 10], [-2, 15], [6, 25]]         ; 9
  $aNum[8][2] = $aTmp
  Local $aTmp = [[-12, 15], [-14, 25], [6, 25]]       ; 6
  $aNum[9][2] = $aTmp
  Return $aNum
EndFunc   ;==>DefineNum

Func IsColor($nColor)
  Return Not _ColorGetRed($nColor) And Not _ColorGetBlue($nColor) And _ColorGetGreen($nColor) > Dec("FA")
EndFunc

$LINE must be the first line (top most) where the bright green color appears
$COLUMN can be located somewhere before
$ENDX can be anywhere right after the 4 digits
$ENDY can be anywhere below under the 4 digits

Tested on your 4 pics and it is working fine (and quite fast under 40ms)

Edited by Nine

“They did not know it was impossible, so they did it” ― Mark Twain

Spoiler

Block all input without UAC

Save/Retrieve Images to/from Text

 Monitor Management (VCP commands)

Tool to search in text (au3) files

Date Range Picker

 Virtual Desktop Manager

Sudoku Game 2020

Overlapped Named Pipe IPC

 HotString 2.0 - Hot keys with string

x64 Bitwise Operations

Multi-keyboards HotKeySet

 Recursive Array Display

Fast and simple WCD IPC

Multiple Folders Selector

 Printer Manager

GIF Animation (cached)

 

Screen Scraping

 

 Multi-Threading Made Easy

 

 

junkew Universalist

junkew

Posted
Posted

ChatGPT nows how to crunch this😀

Upload the pictures
Question: In each picture there are 4 numbers can you recognize them?

Answer:    Here are the numbers I recognize from the images:

  1. Image 1 (pic1.png): The number is 2409.
  2. Image 2 (pic2.png): The number is 0257.
  3. Image 3 (pic3.png): The number is 8035.
  4. Image 4 (pic4.png): The number is 4691.

FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...