Image Search Library

[kangkeng]

Hi Folks,

I'm a great fan of AutoIT. While trying to automate data entry into a java applet based ERP, I found that checking for entry error or success is quite a pain using pixelsearch. Instead, check for a data entry "success" or "failed" image is far simpler.

This a feature found in AHK, I've cut out the necessary code and compiled it into a DLL, and written a au3 library around it. I've tested it in various XP and W2k3 machines, and it seems to work fine. It's a quick and dirty hack I did to get a job done, so your mileage may vary

ImageSearch.zip (84K) Number of downloads:

[ssjdennis]

Thx It working 100%

[Vindicator209]

hey, this rocks, thanks

[A. Percy]

Perfect!

[ssjdennis]

what file type can I use???

[kangkeng]

what file type can I use???

See here...

http://www.autohotkey.com/docs/commands/ImageSearch.htm

Other than GIF, I have not tested the code for any of the other formats. BTW, for lossy image formats you will need to use the variance parameter or the match will likely fail.

[ssjdennis]

See here...

http://www.autohotkey.com/docs/commands/ImageSearch.htm

Other than GIF, I have not tested the code for any of the other formats. BTW, for lossy image formats you will need to use the variance parameter or the match will likely fail.

Thx

[ssjdennis]

I have a problem.

my codes will not work.

#include <ImageSearch.au3>

$x1=0
$y1=0

HotKeySet( "+s", "Start" )
HotKeySet( "+e", "end" )

While 1
   Sleep (100)
WEnd

Func Start()
   While 1
      $result = _ImageSearch("recycle2.bmp",1,$x1,$y1,0)
      if $result=1 Then
         MouseMove($x1,$y1,3)
         MsgBox(0,"Found","Found a empty recycle bin here...")
      EndIf
   WEnd
EndFunc

Func End()
   Exit
EndFunc

and

#include <ImageSearch.au3>

$x1=0
$y1=0

HotKeySet( "+s", "Start" )
HotKeySet( "+e", "end" )

While 1
   Sleep (100)
WEnd

Func Start()
   While 1
      $result = _ImageSearchArea("recycle2.bmp",1,0,0,200,200,$x1,$y1,0)
      if $result=1 Then
         MouseMove($x1,$y1,3)
         MsgBox(0,"Found","Found a empty recycle bin in top left corner")
      EndIf
   WEnd
EndFunc

Func End()
   Exit
EndFunc

but, this code working.

#include <ImageSearch.au3>

$x1=0
$y1=0

$result = _ImageSearch("recycle.bmp",1,$x1,$y1,0)
if $result=1 Then
    MouseMove($x1,$y1,3)
    MsgBox(0,"Found","Found a recycle bin with stuff here...")
EndIf

$result = _ImageSearchArea("recycle.bmp",1,0,0,200,200,$x1,$y1,0)
if $result=1 Then
    MouseMove($x1,$y1,3)
    MsgBox(0,"Found","Found a recycle bin with stuff in top left corner")
EndIf

Plz help me.

Edited March 16, 2008 by ssjdennis

[Zephir]

hey kangkenq,

this is super awesome

very nice work.

keep it up

[kangkeng]

include <ImageSearch.au3>
Func Start()
   While 1
      $result = _ImageSearch("recycle2.bmp",1,$x1,$y1,0)
      if $result=1 Then
         MouseMove($x1,$y1,3)
         MsgBox(0,"Found","Found a empty recycle bin here...")
      EndIf
   WEnd
EndFunc

Haven't tried out your code, but I don't think you should have the While 1 and WEnd in the Func, unless you want to repeatedly show the message box.

[JRSmile]

This is great, dude.

Already included it into my projects.

VERY MUCH THX

[Kickassjoe]

When I first started using AutoIt, this is the type of thing I was looking for, looks nice

[ssjdennis]

Haven't tried out your code, but I don't think you should have the While 1 and WEnd in the Func, unless you want to repeatedly show the message box.

It was a example, but can you help me???

[Zedna]

Looks VERY cool

Five stars from me!

[bf2forlife]

How this _ImageSearchArea works?

$x1=143
$y1=200
_ImageSearchArea("lol.bmp",1,0,0,473,33,$x1,$y1,0)

I wanna search image in area size of 473,33 pixels and is in position 143, 200. But it doesnt work. So whats the problem?

EDIT: Fixed some of the code but still dont work

Edited May 21, 2008 by bf2forlife

[Zedna]

It would be absolutelly awesome to have new optional parameter hWnd

as it is in AutoIt's PixelSearch in latest 3.2.12 version.

In this case search would be done only inside given window/control (and not on whole desktop.).

This way you could search for image also on hidden/minimized/non active windows.

and search could be narrowed on desired control so it will be much faster.

I need such functionality in my current project.

This could be very simple to implement:

Just use GetDC($hWnd)

instead of GetDC(0)

[ygm]

How to use this on a windows? So not on the desktop, but in the windows file browser or something like that.

[Sven]

Thanks for the library, it's way faster than the same thing in EXE format I was using up until now. The exe managed to do 14 images * 10 possibilities in ~75 seconds, the dll one can do the same in 29 seconds.

One important thing (well, at least for me) however is missing from your Autoit code: a specification for a color that can be defined to work as a transparent. This allows for example to search for words, letters or symbols which are displayed over changing backgrounds.

The changed functions:

expandcollapse popup

#include-once
; ------------------------------------------------------------------------------
;
; AutoIt Version: 3.0
; Language:       English
; Description:    Functions that assist with Image Search
;                 Require that the ImageSearchDLL.dll be loadable
;
; ------------------------------------------------------------------------------

;===============================================================================
;
; Description:      Find the position of an image on the desktop
; Syntax:           _ImageSearchArea, _ImageSearch
; Parameter(s):     
;                   $findImage - the image to locate on the desktop
;                   $tolerance - 0 for no tolerance (0-255). Needed when colors of 
;                                image differ from desktop. e.g GIF
;                   $resultPosition - Set where the returned x,y location of the image is.
;                                     1 for centre of image, 0 for top left of image
;                   $x $y - Return the x and y location of the image
;                   $transparency - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of
;                                  the color to be used as transparency; can be omitted if
;                                  not needed
;
; Return Value(s):  On Success - Returns 1
;                   On Failure - Returns 0 
;
; Note: Use _ImageSearch to search the entire desktop, _ImageSearchArea to specify
;       a desktop region to search
;
;===============================================================================
Func _ImageSearch($findImage,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)
   return _ImageSearchArea($findImage,$resultPosition,0,0,@DesktopWidth,@DesktopHeight,$x,$y,$tolerance,$transparency)
EndFunc

Func _ImageSearchArea($findImage,$resultPosition,$x1,$y1,$right,$bottom,ByRef $x, ByRef $y, $tolerance, $transparency=0)
    ;MsgBox(0,"asd","" & $x1 & " " & $y1 & " " & $right & " " & $bottom)
    if not ($transparency = 0) then $findImage = "*" & $transparency & " " & $findImage
    if $tolerance>0 then $findImage = "*" & $tolerance & " " & $findImage
    $result = DllCall("ImageSearchDLL.dll","str","ImageSearch","int",$x1,"int",$y1,"int",$right,"int",$bottom,"str",$findImage)

    ; If error exit
    if $result[0]="0" then return 0
    
    ; Otherwise get the x,y location of the match and the size of the image to
    ; compute the centre of search
    $array = StringSplit($result[0],"|")
   
   $x=Int(Number($array[2]))
   $y=Int(Number($array[3]))
   if $resultPosition=1 then
      $x=$x + Int(Number($array[4])/2)
      $y=$y + Int(Number($array[5])/2)
   endif
   return 1
EndFunc

;===============================================================================
;
; Description:      Wait for a specified number of seconds for an image to appear
;     
; Syntax:           _WaitForImageSearch, _WaitForImagesSearch
; Parameter(s):     
;                   $waitSecs  - seconds to try and find the image
;                   $findImage - the image to locate on the desktop
;                   $tolerance - 0 for no tolerance (0-255). Needed when colors of 
;                                image differ from desktop. e.g GIF
;                   $resultPosition - Set where the returned x,y location of the image is.
;                                     1 for centre of image, 0 for top left of image
;                   $x $y - Return the x and y location of the image
;                   $transparency - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of
;                                  the color to be used as transparency can be omitted if
;                                  not needed
;
; Return Value(s):  On Success - Returns 1
;                   On Failure - Returns 0 
;
;
;===============================================================================
Func _WaitForImageSearch($findImage,$waitSecs,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)
    $waitSecs = $waitSecs * 1000
    $startTime=TimerInit()
    While TimerDiff($startTime) < $waitSecs
        sleep(100)
        $result=_ImageSearch($findImage,$resultPosition,$x, $y,$tolerance,$transparency)
        if $result > 0 Then
            return 1
        EndIf
    WEnd
    return 0
EndFunc

;===============================================================================
;
; Description:      Wait for a specified number of seconds for any of a set of
;                   images to appear
;     
; Syntax:           _WaitForImagesSearch
; Parameter(s):     
;                   $waitSecs  - seconds to try and find the image
;                   $findImage - the ARRAY of images to locate on the desktop
;                              - ARRAY[0] is set to the number of images to loop through
;                                ARRAY[1] is the first image
;                   $tolerance - 0 for no tolerance (0-255). Needed when colors of 
;                                image differ from desktop. e.g GIF
;                   $resultPosition - Set where the returned x,y location of the image is.
;                                     1 for centre of image, 0 for top left of image
;                   $x $y - Return the x and y location of the image
;                   $transparent - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of
;                                  the color to be used as transparent; can be omitted if
;                                  not needed
;
; Return Value(s):  On Success - Returns the index of the successful find
;                   On Failure - Returns 0 
;
;
;===============================================================================
Func _WaitForImagesSearch($findImage,$waitSecs,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)
    $waitSecs = $waitSecs * 1000
    $startTime=TimerInit()
    While TimerDiff($startTime) < $waitSecs
        for $i = 1 to $findImage[0]
            sleep(100)
            $result=_ImageSearch($findImage[$i],$resultPosition,$x, $y,$tolerance,$transparency)
            if $result > 0 Then
                return $i
            EndIf
        Next    
    WEnd
    return 0
EndFuncoÝ÷ ÙìZ^r^jëh×6; find recycle bin if it is in the top left corner of screen
; change 2nd argument to 0 to return the top left coord instead
$result = _ImageSearchArea("recycle.bmp",1,0,0,200,200,$x1,$y1,0,0x000000) ;perfect black used as transparency
if $result=1 Then
    MouseMove($x1,$y1,3)
    MsgBox(0,"Found","Found a recycle bin with stuff in top left corner")
EndIf

This is pieced together to maintain compatibility with my old Imagesearch approach, but works nonetheless

Since this is for games I partially automate, it explains the >for< loop. In games, the HUD (target markers, status bars, ...) have a tendency to blink off and on again so quickly the human eye doesn't notice, but a script surely would. So when an object I'm looking for isn't there, I need to be sure it's not because of a blinking HUD. Ati hardware seems to be more stable than Nvidia on this issue, generating way less HUD blinks, but it's still far from perfect. Tested with Ati x1800 & Nvidia 8800GTS G92 hardware and confirmed by numerous other users.

Edit

@ ygm

Everything your screen shows is "the desktop", be it windows like Firefox or games. Only exclusion, maybe, is video overlays. Haven't yet tested it with those.

Edited July 17, 2008 by Sven

[therms]

Can anyone tell me if this is working for them on Vista?

I cannot get it to work at all. Of course the demo script doesn't work because of Vista's changed icons, but even replacing the included sample images with my own, the function returns a 0 every time.

[comeonyoureds]

Hey, there! First I wanna thank you for porting this function into a dll for usage in AutoIT.

Secondly, I want to ask if it would be possible to make it support the transparency option.Working with objects that don't fit well in a rectangle I think that would be a really nice feature!

From what I understand, the code placed here to support transparency can treat things on the screen as being transparent.I was looking for something that treats part of the image in the file as being transparent.

Or maybe I'm just confused

Edited October 29, 2008 by comeonyoureds