PixelSearch in bitmap (handle to a bitmap, bitmap, or hdc)

[InunoTaishou]

Here's the goal:

• Create a handle to a bitmap object using _WinApi_PrintWindow (this is done and working properly, I can take a screenshot of the window even when it's hidden or offscreen, does not work minimized)
• Be able to search for a pixel color in memory on the handle. I can get it to work if I create a Bitmap from an HBITMAP (_GDIPlus_BitmapCreateFromHBITMAP) and then go through each pixel and check it using _GDIPlus_BitmapGetPixel but it's too slow. I've tried doing the _WinApi_GetPIxel using an $hDC but it's much slower than GDI+ (GD+ takes about 20seconds to search for almost 500,000 pixels, GetPixel takes almost 60seconds)

I found the FastFind library but, honestly, it's sloppy and I'm wanting to search a defined area, not set the starting position and then search the rest of the window. I also found a few examples on the forums but I could not get them to work.

Here's the CaptureWindow function. I'm only wanting to capture the client area (Not the border around the client, hence the -16 for the width and the -38 for the height, gets rid of the title bar and the resize bars on the sides. Also I'm wanting to keep the coordinates relative to the window, so if you only want to capture the screen from 480, 200 to 680, 400, then the rest of the area is black on purpose)

#include-once
Func CaptureWindow(Const $iLeft = 0, Const $iTop = 0, Const $iWidth = -1, Const $iHeight = -1, Const $hWindow = WinGetHandle("[Active]"))
    Local $rect_window = WinGetPos($hWindow)
    Local $hDC = _WinAPI_GetWindowDC($hWindow)
    Local $hDestDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBitmap = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hDestSv = _WinAPI_SelectObject($hDestDC, $hBitmap)
    Local $hSrcDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBmp = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hSrcSv = _WinAPI_SelectObject($hSrcDC, $hBmp)
    
    _WinAPI_PrintWindow($hWindow, $hSrcDC, True)
    
    If ($iWidth > 0 and $iHeight > 0) Then
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $iWidth, $iHeight, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    Else
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $rect_window[2] - 16 - $iLeft, $rect_window[3] - 38 - $iTop, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    EndIf
    
    _WinAPI_SelectObject($hDestDC, $hDestSv)
    _WinAPI_SelectObject($hSrcDC, $hSrcSv)
    _WinAPI_ReleaseDC($hWindow, $hDC)
    _WinAPI_DeleteDC($hDestDC)
    _WinAPI_DeleteDC($hSrcDC)
    _WinAPI_DeleteObject($hBmp)
    
    Return $hBitmap
EndFunc   ;==>CaptureWindow

And the first attempt for PixelSearch, using GDI+ (Fastest)

expandcollapse popup

Func PixelSearchInhBitmap(Const ByRef $hHBmp, Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1)
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBmp)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    If ($iWidth = -1) Then $iWidth = _GDIPlus_ImageGetWidth($hBitmap)
    If ($iHeight = -1) Then $iHeight = _GDIPlus_ImageGetHeight($hBitmap)
    
    Local $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = _GDIPlus_BitmapGetPixel($hBitmap, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000)
    _GDIPlus_BitmapDispose($hBitmap)
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(-1, 0, 0)
    EndIf
EndFunc

Second attempt using _WInApi_GetPixel (Note. I replaced the Autoit function with my own where I replaced the string "gdi32.dll" with a handle to the opened DLL. In the hopes it would improve time. The time difference was not noticable)

expandcollapse popup

Global $HWND_DLL_GDI32 = DLLopen("gdi32.dll")

Func PixelSearchInhDC(Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1, Const $hWnd_window = WinGetHandle("[Active]"))
    Local $hDC = _WinAPI_GetWindowDC($hWnd_window)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $start_time, $end_time
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    If (Not $hDC) Then
        Return SetError(1, 0, 0)
    EndIf
    
    If ($iWidth = -1 or $iHeight = -1) Then
        Local $rect_window = WinGetPos($hWnd_window)
        If ($iWidth = -1) Then $iWidth = $rect_window[2] - $iLeft
        If ($iHeight = -1) Then $iHeight = $rect_window[3] - $iTop
    EndIf
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = __WinAPI_GetPixel($hDC, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000 & "s")
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(1, 0, 0)
    EndIf
EndFunc

Func __WinAPI_GetPixel($hDC, $iX, $iY)
    Local $aRet = DllCall($HWND_DLL_GDI32, 'dword', 'GetPixel', 'handle', $hDC, 'int', $iX, 'int', $iY)
    If @error Or ($aRet[0] = 4294967295) Then Return SetError(@error, @extended, -1)
    ; If $aRet[0] = 4294967295 Then Return SetError(1000, 0, -1)

    Return __RGB($aRet[0])
EndFunc   ;==>__WinAPI_GetPixel

 

Edited January 15, 2016 by InunoTaishou

[InunoTaishou]

Looked into _GDIPlus_BitmapLockBits, this seems to be a bit faster (about 8 seconds faster) but it still takes seconds to do the search.

expandcollapse popup

#include <GDIPlus.au3>
#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
#include <Color.au3>

_GDIPlus_Startup()

TestPixelSearch()

Func TestPixelSearch()
    Local $hImage = _GDIPlus_ImageLoadFromFile(@DesktopDir & "\CaptureWindow.jpg")
    Local $iW = _GDIPlus_ImageGetWidth($hImage), $iH = _GDIPlus_ImageGetHeight($hImage) ;get width and height of the image
    Local $hBitmap = _GDIPlus_BitmapCreateFromScan0($iW, $iH)
    Local $hContext = _GDIPlus_ImageGetGraphicsContext($hBitmap)
    _GDIPlus_GraphicsDrawImageRect($hContext, $hImage, 0, 0, $iW, $iH)

    Local $tBitmapData = _GDIPlus_BitmapLockBits($hBitmap, 0, 0, $iW, $iH, BitOR($GDIP_ILMWRITE, $GDIP_ILMREAD), $GDIP_PXF32RGB)
    Local $iScan0 = DllStructGetData($tBitmapData, "Scan0")
    Local $rbg_color = _ColorGetRGB(0xFFFC00)
    Local $tolerance = 10
    Local $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    Local $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    Local $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    Local $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    Local $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    Local $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    Local $tPixel = DllStructCreate("int[" & $iW * $iH & "];", $iScan0)
    Local $iPixel, $iRowOffset
    Local $found_pixel = False
    Local $abscoord_pixel[2] = [0, 0]

    Local $start_timer = TimerInit()
    For $iY = 0 To $iH - 1
        $iRowOffset = $iY * $iW + 1
        For $iX = 0 To $iW - 1 ;get each pixel in each line and row
            $iPixel = DllStructGetData($tPixel, 1, $iRowOffset + $iX) ;get pixel color
            Local $pixel_color = _ColorGetRGB("0x" & Hex($iPixel, 6))
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $abscoord_pixel[0] = $iX
                $abscoord_pixel[1] = $iY
                $found_pixel = True
                ExitLoop
            EndIf
        Next
    Next
    Local $end_time = TimerDiff($start_timer) / 1000

    MsgBox("", "", $end_time & @CRLF & "Found pixel = " & $found_pixel)

    _GDIPlus_BitmapUnlockBits($hBitmap, $tBitmapData)

    ;cleanup resources
    _GDIPlus_ImageDispose($hImage)
    _GDIPlus_GraphicsDispose($hContext)
    _GDIPlus_BitmapDispose($hBitmap)
    _GDIPlus_Shutdown()
EndFunc

Will probably try to and rewrite this in C++ and give it a test in visual studio, perhaps it will be faster in a dll than in autoit. Also saw something about the WinApi GetDIBits but the apparently copies the whole bitmap into a buffer, not a pointer to a buffer so someone said it would be slow.

Does anyone know of any other way of checking pixels in memory or any way to make this faster? The Autoit Pixelsearch function takes miliseconds to complete. I did find the source code for another *ahem* scripting languages PixelSearch function and it looks like it uses GetDIBits.

[Bert]

Why do you need to do pixel search?

Why not look for the control?

Why also do you need to do a snapshot then look at the picture?

Why not look at the screen directly? 

This seems a real difficult way to solve a problem when there are many simpler and more stable methods to do things. 

[InunoTaishou]

Because it seems fun and an interesting way to learn more about bitmaps. The only problem with that is I don't know a whole lot about bitmaps and my knowledge of the WinApi is very limited. I was also working on a small image editor using GDI+ and being able to find colors in the bitmap will open up the possibility to search and replace colors in a bitmap. I just don't want it to take 20 seconds to go through and replace colors when programs like Photoshop and Gimp can do it in ms and Pixelsearch takes ms to find a pixel on the screen.

That snapshot allows to search in memory, so even if the window is hidden you can still search in it.

Edited January 15, 2016 by InunoTaishou

[Bert]

So your saying you want to make a photo editor? I'm asking for none of your code shows anything to replace colors. It only shows how you want to search a client area. I'm assuming you have a GUI to run the controls for this editor?

Edited January 16, 2016 by Bert

[InunoTaishou]

At this point all I'm trying to do is see if I can quickly, and efficiently, find a color in a bitmap (GDI+ bitmap or a WinAPI handle to a bitmap, since they can be converted easily to each other. I've been using the WinAPI one to capture the client area of a program). My GUI has... bugs in it still so I'm not even using it. Atm I've got 2 scripts on my desktop I've been running trying to test this.

[AndyG]

Hi,

writing a (32-Bit) bitmap into Memory takes a millisecond, writing a struct "over" the RGBA-information (some people say "Pixels" to that ) takes a millisecond too, copying the Information from struct to a string takes another millisecond, searching/replacing "colors" via stringinstr/stringreplace/stringregex[replace] depends on string length, but never more than some milliseconds. 

If you use a slow idea, use slow functions and write slow scripts/programs, its not surprising that you have to wait a long time....