HandleImgSearch (Image Search with ImageSearchDLL embedded)

[lamnhan066]

Today, I will share my work with image search in inactive windows (windows that run in the background) with tolerances (variations).

After a long time of searching and using image search UDF on the internet. I combined all of that with my experienced and created a UDF name HandleImgSearch.

I have embedded the .dll file in this script so it doesn't need any external files, just include it and run. However, this UDF uses a pure AutoIT loop to make the multiple positions returned work so it may be slower if you set the $MaxImg to a high number.

Some highlights:

• Very fast with imagesearchdll embedded, no external files required.
• Tolerances and Max Images are supported.
• Optimized.
• Use with a handle or full screen easily.
• Included global functions, so you can run multiple functions in one capture.
• Examples used for testing included.

Notes:

• Only used to compile to 32-bit AutoIt version (It can completely run on Windows 64-bit)
• The image used for searching should be in a "24-bit Bitmap" format.
• I included the $IsUser32 variable in some places like _GlobalImgInit, _HandleCapture. This variable allowed you to use DllCall with the "PrintWindow" parameter instead of _WinAPI_BitBlt with $SRCCOPY. $IsUser32 = True is useful for the window explorer handle, $IsUser32 = False is useful for the emulator handle (NoxPlayer handle example included).
• If hwnd parameter equals "", it will use the whole screen instead.

Functions:

• Global Functions: Run multiple functions with one capture.
  • _GlobalImgInit: Initialization variables.
  • _GlobalImgCapture: Capture the handle.
  • _GlobalGetBitmap: Get captured bitmap handle.
  • _GlobalImgSearch: This is the main function of this UDF.
  • _GlobalGetPixel: Get pixel color in the captured image.
  • _GlobalPixelCompare: Compare pixel color with captured image pixel color.
• Handle Functions: Capture every time.
  • _HandleImgSearch: This is the main function of this UDF. Search for images in the handle of the window with the tolerance and maximum image options.
  • _BmpImgSearch: Search the picture in the picture instead of the handle.
  • _HandleGetPixel: Get pixel in the handle image.
  • _HandlePixelCompare: Compare color with pixel color of handle image.
  • _HandleCapture: Capture handle screen.

Source code: 

• v1.0.5 (First release): https://github.com/vnniz/HandleImgSearch/releases/tag/v1.0.5
• v2.0.0 (14/04/2023): https://github.com/vnniz/HandleImgSearch/releases/tag/v2.0.0

Thanks to:

• ImageSearchDLL (Author: kangkeng 2008)
• MemoryCall (Author: Joachim Bauch)
• BinaryCall (Author: Ward)

GAMERS - Asking for help with ANY kind of game automation is against the forum rules. DON'T DO IT.

 

 

Edited April 13, 2023 by lamnhan066
Update v2.0.0

[jimmy123j]

Sorry for bother

does any English ver. detail for function Description, Parameters and Return values (fully English ver.)

btw, _HandleImgSearch and _HandleImgWaitExist can search multiple img at same time (as A.png|B.png ... etc.)

thanks for au3 development, it's useful to me

thanks again !!

Edited October 4, 2020 by jimmy123j

[lamnhan066]

On 10/4/2020 at 9:45 AM, jimmy123j said:

does any English ver. detail for function Description, Parameters and Return values (fully English ver.)

I have no English version now. Sorry about that. I will add it later when I have free time and update on this thread.

Some default parameters (You can ask me here if you have another thing make you confused):

• $hwnd: Handle of window you want to find in. If $Handle = "" then it will find images in your current screen.
• $bmpLocal: Path of local image you want to find.
• $x, $y, $iWidth, $iHeight: Area of $hwnd image you want to find in. Default values is full image.
• $MaxImg: Max number of images you want to find (or return).
• $Tolerance: (0 - 100) This is the variation of color you want to find. This parameter is helpfull when you want to share your app with your friend who use other devices.

Some default return values:

• $aCords[0][0]: Total number of positions found.
• $aCords[$i][0]: X coordinate of position $i.
• $aCords[$i][1]: Y cooridinate of position $i.
• $aCords[$i][2]: Bitmap width of position $i.
• $aCords[$i][3]: Bitmap height of position $i.

The width and height of the bitmap are the same as your finding image.

On 10/4/2020 at 9:45 AM, jimmy123j said:

btw, _HandleImgSearch and _HandleImgWaitExist can search multiple img at same time (as A.png|B.png ... etc.)

No, it's now not support this. You can use this by add this a function like:

;~ I have not tested this code yet, but you can use something like this
Func _HandleMultiImgSearch($hwnd, $ImgsPath, $x = 0, $y = 0, $iWidth = -1, $iHeight = -1, $Tolerance = 15, $MaxImg = 1000)
    Local ImgsPathArray = StringSplit($ImgsPath, "|", 2) ;$ImgsPath = 'Path1|Path2|...|PathN'
    Local $Results
    For ImgPath in ImgsPathArray
        $Results = _ArrayAdd($Result, _HandleImgSearch($hwnd, $ImgPath, $x, $y, $iWidth, $iHeight, $Tolerance, $MaxImg))
    Next
    
    Return $Results
EndFunc

;~ Using this func
Local $ImgsPath = 'a.png|b.png|c.png'
Local $Results = _HandleMultiImgSearch($hwnd, $ImgsPath)
ConsoleWrite("> Total of finding images: " & Ubound($Results))
Local $count = 0
For $Coords in $Results
    ;~ An array of positions of each image
    $count += 1
    ConsoleWrite(">> Total of position found of image " & $count & " found: " & Ubound($Coords[0][0]))
Next

 

Edited October 19, 2020 by LTNhanSt94

[jimmy123j]

Sorry for bother, I found the _HandleCapture has some garbled word

that's why I try to use the _HandleCapture, but not working

I think the au3 file download from github had same issue

would you might to update the au3 file again

Thanks !!!

 

au3 download form Github

Github

Edited December 10, 2020 by jimmy123j

[Nine]

2 hours ago, jimmy123j said:

_HandleCapture has some garbled word

No garbled.  It is called ternary conditional operator 

[MaximusCZ]

Hello,

Thanks for this script, altho I have an Issue with it.
When I try to get HandleCapture working, it succesfully makes screenshot of control on given coordinates IF the window is on top. If I put any other window over it, the screenshot actually contains the overlapping window, blocking the window of interest, and inevitably failing finding the covered piece.

Setting "IsUser32" to True just produces black screenshots (of valid resolution tho) no matter if the window is infront or not.

Any idea where to point me? Feels to me like it might be problem with windows composition or something. Tried compiling and running as admin, same result. Win7 Pro, transparency of windows turned off

 

I dont really know what I did different now, but setting IsUser32 to True now works..
thanks for cool script!

Edited December 11, 2020 by MaximusCZ

[match]

hi 

is there anyway to do a colour search in a allocated location 

[Nine]

Yes you can.  You will need to use _GDIPlus_BitmapLockBits (see example) to transfer the bitmap into an 2D array.  You can also use _GDIPlus_BitmapGetPixel to read the color of a specific pixel.

[match]

On 7/19/2021 at 10:03 PM, Nine said:

Yes you can.  You will need to use _GDIPlus_BitmapLockBits (see example) to transfer the bitmap into an 2D array.  You can also use _GDIPlus_BitmapGetPixel to read the color of a specific pixel.

can i use any of the function here 

	; _HandleImgSearch($hwnd, $bmpLocal, $x = 0, $y = 0, $iWidth = -1, $iHeight = -1, $Tolerance = 15, $MaxImg = 1000)
	; _HandleImgWaitExist($hwnd, $bmpLocal, $timeOutSecs = 5, $x = 0, $y = 0, $iWidth = -1, $iHeight = -1, $Tolerance = 15, $MaxImg = 1000)
	; _BmpImgSearch($SourceBmp, $FindBmp, $x = 0, $y = 0, $iWidth = -1, $iHeight = -1, $Tolerance = 15, $MaxImg = 1000)
	; _HandleGetPixel($hwnd, $getX, $getY, $x = 0, $y = 0, $Width = -1, $Height = -1)
	; _HandlePixelCompare($hwnd, $getX, $getY, $pixelColor, $tolerance = 15, $x = 0, $y = 0, $Width = -1, $Height = -1)
	; _HandleCapture($hwnd, $x = 0, $y = 0, $Width = -1, $Height = -1, $IsBMP = False, $SavePath = "", $IsUser32 = False)

[Ranastic]

On 2/17/2020 at 11:18 AM, LTNhanSt94 said:

Today, I will share my work with image search in inactive windows (windows that run in the background) with tolerances (variations).

After a long time of searching and using image search UDF on the internet. I combined all of that with my experienced and created a UDF name HandleImgSearch.

I have embedded the .dll file in this script so it doesn't need any external files, just include it and run. However, this UDF have a feature that will make it run slower is $MaxImg variable, this variable will allow you to return multiple positions of a finding image.

Some highlights:

• Very fast with imagesearchdll embedded, no external files required.
• Tolerances and Max Images are supported.
• Optimized.
• Use with handle or full screen easily.
• Included global functions, so you can run multiple functions in one capture.
• Examples used for testing included.

Notes:

• Only use to compile to 32 bit AutoIt version (It can completely run on Windows 64 bit)
• Image use for searching should be a "24-bit Bitmap" format.
• I included $IsUser32 variable in some places like _GlobalImgInit, _HandleCapture. This variable allowed you to use DllCall with "PrintWindow" parameter instead of _WinAPI_BitBlt with $SRCCOPY. $IsUser32 = True is useful for window explorer handle, $IsUser32 = False is useful for the emulator handle (NoxPlayer handle example included).
• If hwnd parameter equal "", it will use whole screen instead.

Functions:

• Global Functions: Run multiple functions with one capture.
  • _GlobalImgInit: Initialization variables.
  • _GlobalImgCapture: Capture the handle.
  • _GlobalGetBitmap: Get captured bitmap handle.
  • _GlobalImgSearch: This is the main function of this UDF.
  • _GlobalGetPixel: Get pixel color in captured image.
  • _GlobalPixelCompare: Compare pixel color with captured image pixel color.
• Handle Functions: Capture every time.
  • _HandleImgSearch: This is the main function of this UDF. Search for images in the handle of the window with the tolerance and maximum image options.
  • _BmpImgSearch: Search picture in picture instead of handle.
  • _HandleGetPixel: Get pixel in handle image.
  • _HandlePixelCompare: Compare color with pixel color of handle image.
  • _HandleCapture: Capture handle screen.

Source code: https://github.com/ltnhanst94/HandleImgSearch

Thanks to:

• ImageSearchDLL (Author: kangkeng 2008)
• MemoryCall (Author: Joachim Bauch)
• BinaryCall (Author: Ward)

GAMERS - Asking for help with ANY kind of game automation is against the forum rules. DON'T DO IT.

 

 

This isn't game automation ask but i got trouble with your UDF... GlobalImageSearch or HandleImageSearch can't scan my game window with title ""

[Jos]

24 minutes ago, Ranastic said:

can't scan my game window

Care go explain how this is not game related?

Edited September 19, 2021 by Jos

[Ivantu]

Friends, tell me, if I reduce the search area, the coordinates are returned no longer correct, if the default search area is ok.

You can somehow change the search area, get coordinates relative to the size of the control, and not the search area.

I'm sorry for the mistakes in communication, I'm writing through a translator.

[lamnhan066]

9 hours ago, Ivantu said:

Friends, tell me, if I reduce the search area, the coordinates are returned no longer correct, if the default search area is ok.

You can somehow change the search area, get coordinates relative to the size of the control, and not the search area.

I'm sorry for the mistakes in communication, I'm writing through a translator.

In the current version, the result will relate to the search area, you just need to add (x, y) of the result to your search area coordinates to get the results you need.

 

For instance, you need to search in (30, 40, 300, 300) coordinates and get a result (10, 15) then the result you need is (30 + 10, 40 + 15).

[Ivantu]

3 hours ago, lamnhan066 said:

result (10, 15)

This value, as I understand it, is the width and height of the desired image?

[lamnhan066]

5 minutes ago, Ivantu said:

This value, as I understand it, is the width and height of the desired image?

It's a coordinate (left, top) of your search image in the search area.

Edited November 18, 2021 by lamnhan066

[Ivantu]

how can i know these coordinates before starting the search?

[lamnhan066]

2 hours ago, Ivantu said:

how can i know these coordinates before starting the search?

How can you do a specific area searching when you don't know these coordinates? Let's look at this function:

_HandleImgSearch($hwnd, $bmpLocal, $x = 0, $y = 0, $iWidth = -1, $iHeight = -1, $Tolerance = 15, $MaxImg = 1000)

You have to know at least one parameter in $x, $y, $iWidth, $iHeight to search for an area. Let's say you want to search in area [50, 60, 800, 600], so the function will look like:

_HandleImgSearch($hwnd, $bmpLocal, 50, 60, 800, 600)

and the result is `[1, [10, 20, 50, 50]]` which means it has 1 matched possition at [$x, $y, $width, $height] = [10, 20, 50, 50]. If you want to get the relative to your $hwnd coordinates then you need to add [10 + 50, 20 + 60] = [60, 80].

In addition, if you want to get the relative to your current whole screen coordinates, you need to use WinGetPos to get your current $hwnd coordinates and add the [x, y] coordinates to it like the above.

[Ivantu]

Thank you, I will try to figure it out, thanks again so much for the answer!

[Ivantu]

Please tell me where I can change the _Handle Img Search function so that it returns the coordinates I need?

I don 't want to just prescribe it manually every time )

Edited November 19, 2021 by Ivantu

[lamnhan066]

On 11/19/2021 at 12:54 PM, Ivantu said:

Please tell me where I can change the _Handle Img Search function so that it returns the coordinates I need?

I don 't want to just prescribe it manually every time )

I'm only bring the best way to get the results because the UDF can return multiple positions, so you can modify the UDF yourself to get the result that you want to or just write a function to parse the results, the second way may be easier to do because it won't break the core function of the UDF.

Edited November 25, 2021 by lamnhan066