Another bitmap brute-forcing thread

[junkew]

31 minutes ago, Hashim said:

This looks really promising but unfortunately I can't seem to reproduce it, are you sure your version is returning good values? When I run it all outputs return 1111 again, although I've experimented with every single posx and posy that's even close. I added screenshot functionality to debug the capture and it looks like it should definitely be close to there. Any idea what might be going on here?

 

Your 4 is cut to early. Make sure your numbers all are same width. The matrix for 1 digit should fit 3 times for three numbers.

[junkew]

Your 2nd picture you can even see better you are off a few pixels as at the right I can see pixels of the digit thats not visible.

[Werty]

Hmm, the image you posted is one pixel off, should look like this...

That said, I dont have much experience with automation, I use autoit for other things, so I might have assumed wrong that WinWaitActive should give the correct handle to the tools windows, try something like $tools = WinGetHandle(blablabla) or something, as the script should give you atleast other result than 1111 when you are so close.

Test yourself how to get the correct handle to the tools windows, or have someone with more automation experience chime in.

I get my handle to my test gui where it works like this....

$gui = GUICreate("Getnumber", 320, 240, -1, -1)

Maybe something like this...

$hTool = Run("tool.exe")

;and then change the GetWindowDC accordingly.

Edited Monday at 08:35 PM by Werty

[junkew]

Just hover with au3inf over the window or control and post that info. It will help a lot if we know the name, type of control.

[Werty]

51 minutes ago, Hashim said:

are you sure your version is returning good values?

Very sure

If needed/wanted I can post one of my test examples later where you can see it works, right now I have to make dinner, so I'll be in and out for the next couple of hours.

Edited Monday at 08:45 PM by Werty

[Hashim]

3 hours ago, Werty said:

Very sure

If needed/wanted I can post one of my test examples later where you can see it works, right now I have to make dinner, so I'll be in and out for the next couple of hours.

No need for that, I trust you have everything right, and I agree it's really weird that not even a single output is returning anything other than 1111s. I now have GetHandle working but that doesn't seem to make a difference, so I'll just have to think it through. Feel free to enjoy your dinner in peace, there's no rush, this is all theoretical at this point anyway

Edited Monday at 11:56 PM by Hashim

[Hashim]

54 minutes ago, junkew said:

Just hover with au3inf over the window or control and post that info. It will help a lot if we know the name, type of control.

This is how I got my current set of coords as well as minusing the coords from the previous version of the script from the program's window area, both result in basically the same range of numbers, those are obviously more or less correct since the screenshot is not miles away, but I've tried almost all combinations of those numbers, so I'm guessing the issue is probably elsewhere. Just in case though, I'm going to manually try every single one of those numbers in combination with each other. Seems like taking this script to the next level will have to involve brute-forcing the coordinates themselves. If none of these still return values other than 1111 then I can't see how the positioning could be the problem.

Edited Monday at 09:26 PM by Hashim

[Hashim]

1 hour ago, junkew said:

Your 2nd picture you can even see better you are off a few pixels as at the right I can see pixels of the digit thats not visible.

This particular tool has 4 digits but it seems the width and height for the capture (88, 32) have only ever included the first 3 digits, I'm not exactly sure how it works but it successfully recognised 1 million outputs so something is obviously working and I don't think that could be the problem either.

[Werty]

16 hours ago, Hashim said:

the width and height for the capture (88, 32) have only ever included the first 3 digits

It's actually including all four, the fourth is just only the first column of that number, as that is all we need.

I'll see, as i promised in the other thread, about making a short tutorial/explanation of how it works, this time I promise I will get it done.....after dinner.

Edited Tuesday at 01:41 PM by Werty

[junkew]

• 4 digits is 10.000 numbers, why try 1,000,000 times?
• Speed optimization. Why? In general its fun to try in AutoIt but if speed is needed switch to a compiled language
  • _WinAPI_HashString($stringforDigit,false,2) to get a 2 byte hash, easier to compare, no clue if its faster
• Most likely a boundary issue?
  88*32 =2816 pixels then it depends on the goal of the for loop to 2816 with step 440 if you want to hit 7 or 8 bits
  1 441 881 1321 1761 2201 2602 (3003)
  Not sure if 7 or 8 bits make a difference in the first column of a digit. I assume 7 would be enough to have 4 digits in 28 comparisons
• In my example demo code I scale down to 8x8 bits
• Why dinner first? 😉
• ...

Edited Monday at 10:55 PM by junkew

[junkew]

And a small explainer on what @Werty is doing (I am off some pixels but I hope @Hashim get the concept as indicated by the red dots)

Zoom in on the image to see the 28 pixels used for comparison and understanding why its important to have the screenshots exactly aligned.

Edited Monday at 11:13 PM by junkew

[Hashim]

46 minutes ago, junkew said:

• 4 digits is 10.000 numbers, why try 1,000,000 times?
• Speed optimization. Why? In general its fun to try in AutoIt but if speed is needed switch to a compiled language
  • _WinAPI_HashString($stringforDigit,false,2) to get a 2 byte hash, easier to compare, no clue if its faster
• Most likely a boundary issue?
  88*32 =2816 pixels then it depends on the goal of the for loop to 2816 with step 440 if you want to hit 7 or 8 bits
  1 441 881 1321 1761 2201 2602 (3003)
  Not sure if 7 or 8 bits make a difference in the first column of a digit. I assume 7 would be enough to have 4 digits in 28 comparisons
• In my example demo code I scale down to 8x8 bits
• Why dinner first? 😉
• ...

There are lots of repeats in the outputs, not really a problem, I've just been tasked with creating a database for all 1M inputs along with their outputs but actually I've already done that and this is optimising for speed for other versions of the tool that have 7, 8 and 9 digit inputs. You're probably right that I'm focusing on the wrong things and should be using a compiled language that will be even faster than the current AutoIT solution, although I'm not sure how much faster since Werty's <1ms is pretty fast and I suppose both would be calling the underlying Windows APIs and DLLs anyways?

Edited Monday at 11:30 PM by Hashim

[Werty]

Here's the pic from the first thread....

Imagine the first column of each number is a BYTE, with Most Significant Bit being 128, then 64, 32 16 8 4 2 1, we dont need the 128 as each number is only 7 imaginary bits tall, then the loop starts at the top left corner and reads the first bit, if it's lit it adds $value=64 to $code[0], if unlit nothing is added, then the next DllStructGetData reads the next number, and so on, when it gets to the end (last number) then $value is divided by 2 so it's 32 which is the next significant bit and the STEP in the loop goes to next line/bit, if it finds any lit bits it adds $value to $code, and so on....

The $code's now have values, which we then look up in the lookup table, like 114 is 5, the last in the lookup as it's the largest number, while 4 is located at 12'th place.

 

Edited Thursday at 01:17 AM by Werty

[junkew]

@Werty tried to run your example but only get 1111 as a result for 9.png see below my reproducing script.

Could not directly see why it gives 1111 instead of 9142 or was this about getting the exact $posx and $posy right
Maybe I am miscalculating 100,100 for the form +16,16 for the drawImageRect.

 

expandcollapse popup

#include <GDIPlus.au3>
#include <WinAPI.au3>
HotKeySet("{ESC}", "_exit")


;Be sure $posx and $posy is pointing at the correct spot!
Global $posx = 116 + 29 + 3, $posy = 116 + 32 + 12 , $result = "", $code[4], $capture, $pixels = DllStructCreate('dword[2816]')
;Lookup table to avoid searching
Global $digit[115] = [1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,6,0,0,0,3,2,0,0,0,0, _
                      0,0,0,0,0,0,0,0,9,0,0,0,0,0,8,0,0,0,0,0, _
                      0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,5]
;Test section
;--------------------------------------------------------------
$gui = GUICreate("Getnumber", 640, 480, 100, 100)
       GUISetState()
       WinWaitActive($gui); <--- necessary or screencapture fails

            _GDIPlus_Startup()
$graphics = _GDIPlus_GraphicsCreateFromHWND($gui)
   $image = _GDIPlus_BitmapCreateFromFile(@UserProfileDir & "\downloads\examples\" & "9.png")
            _GDIPlus_GraphicsDrawImageRect($graphics, $image, 16, 16, 176, 56)
Sleep(250); <------- necessary because drawimagerect() is too slow, script occasionally fails without
;--------------------------------------------------------------
;/Test section

;~  $hDDC = _WinAPI_GetDC(0)
    ;should be
;~  $hDDC = _WinAPI_GetWindowDC($tool)
    $hDDC = _WinAPI_GetDC($gui)

    $hCDC = _WinAPI_CreateCompatibleDC($hDDC)
 $capture = _WinAPI_CreateCompatibleBitmap($hDDC, 88, 32)
            _WinAPI_SelectObject($hCDC, $capture)
            _WinAPI_BitBlt($hCDC, 0, 0, 88, 32, $hDDC, $posx, $posy, 0x00CC0020)
            _WinAPI_DeleteDC($hCDC)
Sleep(250)
local $t2 = TimerInit()
$result = Getnumber($result)
Consolewrite("Time: " & TimerDiff($t2) & @crlf)
Consolewrite("Result: " & $result & @crlf)

While 1
   Sleep(10)
WEnd

Func Getnumber($result)

    Local $code[4] = [0,0,0,0], $value = 64
    DllCall('gdi32.dll', 'dword', 'GetBitmapBits', 'ptr', $capture, 'dword', DllStructGetSize($pixels), 'ptr', DllStructGetPtr($pixels))

For $loop = 1 To 2816 Step 440
    $code[0] += DllStructGetData($pixels, 1, $loop   ) > 4278190080 ? $value:0
    $code[1] += DllStructGetData($pixels, 1, $loop+29) > 4278190080 ? $value:0
    $code[2] += DllStructGetData($pixels, 1, $loop+58) > 4278190080 ? $value:0
    $code[3] += DllStructGetData($pixels, 1, $loop+87) > 4278190080 ? $value:0
    $value /= 2
Next
;~ consolewrite($code[0] & $code[1] & $code[2] & $code[3])
    Return String($digit[$code[0]]) & String($digit[$code[1]]) & String($digit[$code[2]]) & String($digit[$code[3]])
EndFunc

Func _exit()
    _GDIPlus_Shutdown()
    Exit
 EndFunc

 

[Werty]

@junkew, yes, the posx and posy must be EXACTLY on the correct spot (red pixel).

 

Edited Tuesday at 06:00 AM by Werty

[Nine]

Maybe my code is simpler to use ?

[Werty]

@junkew, I fixed your script, I tried and tried with no result until I noticed you changed GetWindowDC() back to GetDC(), that doesnt work, it has to be GetWindowDC().

(though I have no idea why getdc($gui) doesnt work, I agree it should work somehow, I tried adding $CAPTUREBLT to the bitblt, with no result)

expandcollapse popup

#include <GDIPlus.au3>
#include <WinAPI.au3>
HotKeySet("{ESC}", "_exit")


;Be sure $posx and $posy is pointing at the correct spot!
Global $posx = 37+16, $posy = 38+16 , $result = "", $code[4], $capture, $pixels = DllStructCreate('dword[2816]')
;Lookup table to avoid searching
Global $digit[115] = [1,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,6,0,0,0,3,2,0,0,0,0, _
                      0,0,0,0,0,0,0,0,9,0,0,0,0,0,8,0,0,0,0,0, _
                      0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, _
                      0,0,0,0,0,0,0,0,0,0,0,0,0,0,5]
;Test section
;--------------------------------------------------------------
$gui = GUICreate("Getnumber", 640, 480, 100, 100)
       GUISetState()
       WinWaitActive($gui); <--- necessary or screencapture fails

            _GDIPlus_Startup()
$graphics = _GDIPlus_GraphicsCreateFromHWND($gui)
   $image = _GDIPlus_BitmapCreateFromFile("3.png")
            _GDIPlus_GraphicsDrawImageRect($graphics, $image, 16, 16, 176, 56)
Sleep(250); <------- necessary because drawimagerect() is too slow, script occasionally fails without
;--------------------------------------------------------------
;/Test section

;~  $hDDC = _WinAPI_GetDC(0)
    ;should be
 $hDDC = _WinAPI_GetWindowDC($gui)
;~     $hDDC = _WinAPI_GetDC($gui)

    $hCDC = _WinAPI_CreateCompatibleDC($hDDC)
 $capture = _WinAPI_CreateCompatibleBitmap($hDDC, 88, 32)
            _WinAPI_SelectObject($hCDC, $capture)
            _WinAPI_BitBlt($hCDC, 0, 0, 88, 32, $hDDC, $posx, $posy, 0x00CC0020)
            _WinAPI_DeleteDC($hCDC)
Sleep(250)
local $t2 = TimerInit()
$result = Getnumber($result)
Consolewrite("Time: " & TimerDiff($t2) & @crlf)
Consolewrite("Result: " & $result & @crlf)

While 1
   Sleep(10)
WEnd

Func Getnumber($result)

    Local $code[4] = [0,0,0,0], $value = 64
    DllCall('gdi32.dll', 'dword', 'GetBitmapBits', 'ptr', $capture, 'dword', DllStructGetSize($pixels), 'ptr', DllStructGetPtr($pixels))

For $loop = 1 To 2816 Step 440
    $code[0] += DllStructGetData($pixels, 1, $loop   ) > 4278190080 ? $value:0
    $code[1] += DllStructGetData($pixels, 1, $loop+29) > 4278190080 ? $value:0
    $code[2] += DllStructGetData($pixels, 1, $loop+58) > 4278190080 ? $value:0
    $code[3] += DllStructGetData($pixels, 1, $loop+87) > 4278190080 ? $value:0
    $value /= 2
Next
;~ consolewrite($code[0] & $code[1] & $code[2] & $code[3])
    Return String($digit[$code[0]]) & String($digit[$code[1]]) & String($digit[$code[2]]) & String($digit[$code[3]])
EndFunc

Func _exit()
    _GDIPlus_Shutdown()
    Exit
 EndFunc

 

Edited Tuesday at 12:24 PM by Werty

[junkew]

12 hours ago, Nine said:

Maybe my code is simpler to use ?

Yes offcourse 🙃🤫

[Nine]

24 minutes ago, junkew said:

Yes offcourse

Understand.  Why do simple when you can do complicated...

[Werty]

11 minutes ago, Nine said:

complicated

Oh, didnt know my code was complicated, thanks for letting me know.