Jump to content

Recommended Posts

Posted

Just tested this out and it works great.

Little disappointed that it lacks transparency though. I understand it's a work in progress.

Hope to see the ToDo list completed.

(Would be nicer if the code in the first post was formatted and was in the autoit bbcode just for the sake of readablity)

  • 2 months later...
Posted

I tried running the example and search however i get the following error

---------------------------
AutoIt Error
---------------------------
Line 84  (File "C:\Users\RA029440\Desktop\Misc\imagesearch\bmp\BMPexample.au3"):
$tResult=findBMP($BMP1,$BMP2, $Bool)
$tResult=^ ERROR
Error: Unknown function name.
---------------------------
OK  
---------------------------

Does anyone know why? Do i have to name the files from extraction something else? how about the search function itself? and the example file?

  • 2 months later...
Posted (edited)

I tested your function "findBMP()" I saved it to the file "_FindBMP.au3"

and then I used the below code to test it, and it works, the items I look for on my screen I have already generated .bmp files for. so I do not need to do that when I run the script, I just need to look for them.

#RequireAdmin;win7
#include <GDIPlus.au3>
#Include <ScreenCapture.au3>
#Include <_FindBMP.au3>
;hotkey to run function to find image on screen
HotKeySet("{F11}","Fpic")

; If debugging output is wanted to understand and analyse the algorithm put this on true/false
Const $bDebug=False
Const $cMatchLinePercentage=0.99 ;Minimum matching, put on 1 if exact matching is needed on all lines

;Constants for type of picture matching
const $c24RGBFullMatch=1 ;Load as 24 bits and full match
const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match
const $c16RGBFullMatch=3 ;Load as 16 bits and full match
const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match

$fscreen = @scriptDir & "\fscreen.bmp"

$find1 = @scriptDir & "\item1.bmp"
$find2 = @scriptDir & "\item2.bmp"
$find3 = @scriptDir & "\item3.bmp"
$find4 = @scriptDir & "\item4.bmp"
$find5 = @scriptDir & "\item5.bmp"

while 1
    sleep(1000)
wend

Func Fpic()
    _GDIPlus_Startup()
    
    _ScreenCapture_Capture($fscreen, 0, 0, -1, -1, False)   
    $tResult = findBMP($fscreen, $find1, $c24RGBFullMatch)
    MsgBox(4096, "Test", "result: "&$tResult)
    
    _GDIPlus_Shutdown()
EndFunc

I am very interested in getting HBITMAP to work for the full screen capture, because then it does not need to be written to disk which is much faster.

I tried what I though would work but it did not:

$fscreen = _ScreenCapture_Capture("", 0, 0, -1, -1, False);HBITMAP method

does anyone know how to compare a .bmp file to an HBITMAP?

solution can be an example or something quick and dirty and I can make it work from there, much appreciated. (just not sure where to look next.)

Edited by xekon
  • 4 months later...
Posted

To find on the screen I checked the library which just is doing

 

If $BMPFile="SCREEN" Then
        $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
So you just can do

$tResult = findBMP("SCREEN", $find1, $c24RGBFullMatch)
No need to first save to disk (I also have doubt if that is really slow nowadays as it will be most likely cached) 
  • 7 months later...
  • 1 year later...
Posted

old thread but awesome script . may i ask a question hope junkew still flowing this thread or some one could help me with this code .

so this script will help me to detect un object (bmp) on other bmp and it return the coordinate where it founded , this is very good job . but i want it to find all matches when the object exists in image many times not to find it only once  thank you in advance .

Posted
Posted

Checked and updated first post

  • Works with W10 (and earlier) and AutoIt 3.3.14.2  (and earlier)
  • Calculator controls are not recognized so hardcoded that for this example in first post
  • To recognize controls use FAQ 31 this post is about recognizing bitmap in other bitmap
  • No fixes in the algorithm itself it was mainly not working on the calculator as these calc controls are not recognized by AutoIt
  • 9 months later...
  • 1 year later...
Posted

hi @junkew i try to use yout program for find in foto other foto but probably  i  understund  nothing or less ,:D i try to simplify your  example because is  just complicated for me

i create this script simply

#cs ----------------------------------------------------------------------------

 AutoIt Version: 3.3.14.5
 Author:         myName

 Script Function:
    Template AutoIt script.

#ce ----------------------------------------------------------------------------

; Script Start - Add your code below here
Opt('MustDeclareVars', 1)

#include <GDIPlus.au3>
#Include <ScreenCapture.au3>
#include <string.au3>
#include <WinAPI.au3>
#include <WindowsConstants.au3>

Const $cMatchLinePercentage=0.99    ;Minimum matching, put on 1 if exact matching is needed on all lines

;~ Constants for type of picture matching
const $c24RGBFullMatch=1    ;Load as 24 bits and full match
const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match
const $c16RGBFullMatch=3    ;Load as 16 bits and full match
const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match

Global $begin,$BitmapData,$pixelFormat



Local $OriginalFoto= @ScriptDir & "\source.bmp"
Local $FototoSearch= @ScriptDir & "\tofind.bmp"



findBMP($OriginalFoto, $FototoSearch, $c24RGBFullMatch)




;===============================================================================
; Function Name:    findBMP
; Description:    Finds a bitmap (.BMP) in another BMP file (other formats PNG and TIFF work also other formats less usefull GIF, JPEG, Exif, WMF, and EMF should work)
; Syntax:          findBMP($BMP1, $BMP2, $MatchType=TRUE)
;
; Parameter(s):  $BMP1           = Filename of bitmap to search in
;                  $BMP2             = Filename of bitmap to search for
;                  $MatchType       = c24RGBFullMatch, c24RGBPartialMatch, c16RGBFullMatch, c16RGBPartialMatch
;
; Return Value(s):  On Success:   = Returns Array List
;                  On Failure:   = @error 1 (Control was found but there was an error with the DLLCall)
;
; Author(s):        JunkEW
;
; Note(s):
;               * Its never an exact match even with TRUE as last few bits are disposed in algorithm and lines below
;                are not checked under assumption that those are 99.99% of the time correct
;              * locking bits overview http://www.bobpowell.net/lockingbits.htm
; ToDo:
;               * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html
;               * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line)
;               * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen easier)
; Example(s):
;
;===============================================================================

Func findBMP($BMP1, $BMP2, $MatchType=$c24RGBFullMatch)
    Dim $fLine[1];Line number of found line(s), redimmed when second picture size is known
    Dim $BMP1Data="", $BMP1Width=0, $BMP1Height=0, $BMP1LineWidth=0;
    Dim $BMP2Data="", $BMP2Width=0, $BMP2Height=0, $BMP2LineWidth=0
    Dim $foundAt = "", $matchPossible=FALSE, $matchedLines=0, $foundAtLeft=-1, $foundAtTop=-1
    Dim $bestMatchLine=-1, $HighestMatchingLines=-1; For knowing when no match is found where best area is
    Dim $iPos=1;
    dim $imgBytes, $avData;

    local $iFuzzyDist, $searchFor, $iAbove, $bMatchPossible, $aboveLine
    local $j, $imgBits
    local $tDebug

    if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c24RGBPartialMatch) then
        $imgBytes=3
    Else
        $imgBytes=2
    endif

; Load the bitmap to search in
    getImage($BMP1, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes)
    $BMP1Data = BinaryToString($BMP1Data)

; Load the bitmap to find
    getImage($BMP2, $BMP2Data, $BMP2Width, $BMP2Height, $BMP2LineWidth, $imgBytes)
;Make it strings to be able to use string functions for searching
    $BMP2Data = BinaryToString($BMP2Data)
#cs
if $bDebug=True Then
;~ START debugging
;Get some debugging information
    FileDelete(@ScriptDir & "\BMP1DATA.TXT")
    FileDelete(@ScriptDir & "\BMP2DATA.TXT")
    FileDelete(@ScriptDir & "\COMPAREDLINES.TXT")

    consolewrite($BMP1Width & @crlf)
    consolewrite($BMP1Height & @crlf)
    consolewrite($BMP1LineWidth & @crlf)
    consolewrite(stringlen($BMP1Data) & @crlf)

    consolewrite(".{" & $BMP1LineWidth & "}"& @CRLF)

    consolewrite(".{" & $BMP2LineWidth & "}"& @CRLF)
    $tDebug=StringRegExpReplace(_stringtohex($BMP1Data),"(.{" & $BMP1LineWidth*2 & "})","$1" & @CRLF)
    consolewrite(@error & @extended & @CRLF)
    FileWrite (@ScriptDir & "\BMP1DATA.TXT" , $tdebug)

    $tDebug=StringRegExpReplace(_stringtohex($BMP2Data),"(.{" & $BMP2LineWidth*2 & "})","$1" & @CRLF)
    consolewrite(@error & @extended & @CRLF)
    FileWrite (@ScriptDir & "\BMP2DATA.TXT" , $tdebug)
;~ END debugging
EndIf
#ce
;For reference of line where in BMP2FindIn a line of BMP2Find was found
    If $BMP2Height = 0 Then
        SetError(1,0,0)
        Return  False
    EndIf

    ReDim $fline[$BMP2Height]

;If exact match check every 1 line else do it more fuzzy (as most likely other lines are unique)
    if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c16RGBFullMatch) Then
        $iFuzzyDist = 1
    Else
;Check fuzzy every 10% of lines
        $iFuzzyDist = ceiling(($bmp2height * 0.1))
    endIf

    $begin = TimerInit()
;Look for each line of the bitmap if it exists in the bitmap to find in
;~ Split bitmap to search in lines
;~  $avData=stringregexp($BMP2Data, ".{1," & $BMP2lineWidth & "}+", 3)
dim $searchForRegEx
$searchForRegEx = StringMid($BMP2Data, 1 + (0 * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
$searchForRegEx = $searchForRegEx & ".*" & StringMid($BMP2Data, 1 + ($BMP2Height * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
Local $aArray=stringregexp($BMP1Data,$searchForRegEx,3)
if @error=0 Then
    consolewrite("Regex found " & TimerDiff($begin) & " ms " & @crlf)
;~ For $i = 0 To UBound($aArray) - 1
;~     consolewrite("RegExp Test with Option 3 - " & $i & $aArray[$i])
;~ Next
endif


    For $i = 0 To $BMP2Height - 1
;Minus imgbytes as last bits are padded with unpredictable bytes (24 bits image assumption) or 2 when 16 bits
        $searchFor = StringMid($BMP2Data, 1 + ($i * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
;~      $searchfor=stringleft($avData[$i],$BMP2lineWidth - $imgBytes)

        $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos)
;       $iPos = StringInStr($BMP1Data, $searchFor)


;Look for all lines above if there is also a match
;Not doing it for the lines below as speed is more important and risk of mismatch on lines below is small
        $iAbove=1
        if $iPos > 0 then
            $bMatchPossible=True
            $matchedLines=1;As first found line is matched we start counting
;Location of the match
            $foundAtTop = Int($iPos / $BMP1lineWidth) -$i
            $foundAtLeft =  int(mod($iPos,$bmp1linewidth) / $imgBytes)
        Else
;~          No 1st line found so far nothing to start matching
            $bMatchPossible=false
            exitloop
        endif

        while (($i+$iAbove) <= ($BMP2Height -1)) and ($bMatchPossible=True)
            $searchFor = StringMid($BMP2Data, 1 + (($i + $iAbove) * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
;~          $searchfor = stringleft($avData[$i+$iAbove],$BMP2lineWidth - $imgBytes)
            $aboveLine = stringmid($BMP1Data,$iPos + ($iAbove * $BMP1LineWidth), ($BMP2LineWidth - $imgBytes))
#cs
            if $bDebug=True Then
            ;~ START debugging
                $tDebug=StringRegExpReplace(_stringtohex($searchfor),"(.{8})","$1_")
                FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf)
                $tDebug=StringRegExpReplace(_stringtohex($aboveline),"(.{8})","$1_")
                FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf)
                If $aboveLine <> $searchFor Then
                    FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "** MISMATCH ** of line " & (1+$iAbove) & " in 2nd bitmap at line " & ($foundAtTop + $i) & @crlf)
                endIf
            ;~ END debugging
            EndIf
#ce
            if comparePicLine($aboveline,$searchfor) = false then
                $bMatchPossible=False
;To remember the area with the best match
                if $matchedLines >= $HighestMatchingLines Then
                    $HighestMatchingLines = $matchedLines

    ;Best guess of location
;~                  $foundAtTop = $fline[$i] + $i - $BMP2Height
                    $foundAtTop = Int($iPos / $BMP1lineWidth);+ $i - $BMP2Height
                    $bestMatchLine = Int($iPos / $BMP1lineWidth)
                EndIf
                ExitLoop
            EndIf
            $matchedLines=$matchedLines + 1
            $iAbove=$iAbove+$iFuzzyDist
        WEnd

;If bMatchPossible is still true most likely we have found the bitmap
        if $bmatchPossible = True then
;~          ConsoleWrite("Could match top: " & $foundAtTop & " left: " & $foundAtLeft & " in " & TimerDiff($begin) / 1000 & "  seconds" & @LF)
;           MouseMove($foundatleft,$foundatTop)
            exitloop
        else
;~          consolewrite("i not matched " & $ipos & " " & $matchedlines & @crlf )
        EndIf

    Next

;For some debugging of time
;   if $bMatchPossible = True Then
;       ConsoleWrite("Searching took " & TimerDiff($begin) / 1000 & "  seconds " & @LF)
;   Else
;       ConsoleWrite("NOT FOUND Searching took " & TimerDiff($begin) / 1000 & "  seconds" & @LF)
;   endif

;Return an error if not found else return an array with all information
    if $bMatchPossible = False Then
        SetError(1, 0, 0)
    endif
;   return stringsplit($bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine,";")
    return $bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine
EndFunc;==>findBMP

func comparePicLine($s1,$s2)
    local $sLen, $iMatched
    if $s1=$s2 Then
        return True
    Else
        $iMatched=0
        $sLen=stringlen($s1)
        for $tJ=1 to $slen
            if stringmid($s1,$tJ,1)=stringmid($s2,$tJ,1) Then
                $iMatched=$imatched+1
            Else
                #cs
                if $bDebug=True Then
                ;~ START debugging
                    FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "Debug mismatch pixel " & $tj & "=" & int($tj/4) & @crlf)
                endif
                #ce
            endif
        Next
        if  ($iMatched / $sLen ) > $cMatchLinePercentage then
            return true
        Else
            return false
        endif
    EndIf
endfunc

Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3)
local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle

; Load the bitmap to search in
    If $BMPFile="SCREEN" Then
        $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
;try to get a handle
        $handle = WinGetHandle($BMPFile)
        If @error Then
;Assume its an unknown handle so correct filename should be given
            $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)
        Else
            $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False)
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
        EndIf
    EndIf

;Get $tagGDIPBITMAPDATA structure
;~  ConsoleWrite("Bitmap Width:    " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
;~  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison
    if ($imgBytes=2) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    Else
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    endIf

    If @ERROR Then MsgBox(0,"","Error locking region " & @error)

    $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap.
    $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap.

    $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData")

    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose ($pBitmap)
    _WinAPI_DeleteObject ($pBitmap)

EndFunc;==>GetImage
; Draw rectangle on screen.
Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
    Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2
    $x1 = $tLeft
    $x2 = $tRight
    $y1 = $tTop
    $y2 = $tBottom
    $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop)
    $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color)
    $obj_orig = _WinAPI_SelectObject($hDC, $hPen)

    _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right
    _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right
    _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right
    _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left

    ; clear resources
    _WinAPI_SelectObject($hDC, $obj_orig)
    _WinAPI_DeleteObject($hPen)
    _WinAPI_ReleaseDC(0, $hDC)
EndFunc   ;==>_UIA_DrawRect

but tell me error looking region 1  2  time o_?_O

 

  • 2 weeks later...
Posted

your error information is very cryptic.

The whole algorithm just tries to find a matching image string in another imagestring and if it finds one it starts to match more lines.

If not all lines have a match you get a fuzzy percentage where you can decide on if its match or not match.

Its just quicker then doing pixel by pixel compare in AutoIt. 

 

  • 3 weeks later...
Posted
On 11/24/2019 at 7:24 PM, junkew said:

your error information is very cryptic.

The whole algorithm just tries to find a matching image string in another imagestring and if it finds one it starts to match more lines.

If not all lines have a match you get a fuzzy percentage where you can decide on if its match or not match.

Its just quicker then doing pixel by pixel compare in AutoIt. 

 

Hey,

correct me if I'm wrong please, but fuzz variable doesn't work on the first line of the image u are searching for, so if the first line doesn't have exact match whole script returns False without even checking any more lines right?

Is there a way to add the fuzz into the first line of the search as well?  $searchFor variable is i believe the first line (that must be exact match) of image you want to search for regardless of what constant for type of picture matching u choose.

Otherwise its very neat, easy to use and very fast UDF.

Thank you for sharing and updating this. 👍

  • 2 years later...
Posted

It still works nicely under W10, modified a little so win10 calc example works completely with bitmap searching (but make sure your bitmap matches in detail so it really makes a difference if you have your window active or not have bitmap with red parts on it all small little differences will lead to a mismatch)

#Region includes
Opt('MustDeclareVars', 1)
;#include <GUIConstants.au3>
#include <GDIPlus.au3>
#Include <ScreenCapture.au3>
#include <string.au3>
#include <WinAPI.au3>
#include <WindowsConstants.au3>
#EndRegion

;~ If debugging output is wanted to understand and analyse the algorithm put this on true/false
;~ Const $bDebug=True
Const $bDebug=true

#region example
;~ Const $cMatchLinePercentage=0.97 ;Minimum matching, put on 1 if exact matching is needed on all lines
Const $cMatchLinePercentage=0.99    ;Minimum matching, put on 1 if exact matching is needed on all lines

;~ Constants for type of picture matching
const $c24RGBFullMatch=1    ;Load as 24 bits and full match
const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match
const $c16RGBFullMatch=3    ;Load as 16 bits and full match
const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match

; ** Example start **
; Screen samples
Global Const $Bitmap1Filename = @scriptDir & "\FULLSCREEN.BMP"
Global Const $Bitmap2Filename = @scriptDir & "\CALCULATOR.BMP"
Global Const $Bitmap3Filename = @scriptDir & "\BACKSPACE.BMP"

;BMP samples
local $calcHWND, $begin, $pos, $aWinPos, $aWinCSize, $start
local $pBitmap, $BitmapData, $pixelFormat
Opt("WinTitleMatchMode", 4);Matching windows by advanced options

; Initialize GDI+ library
_GDIPlus_Startup ()

;Calculator, make sure its started
$calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]")
If $calcHWND = "" Then
Run("calc.exe")
Sleep(2000)
$calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]")
EndIf

If $calcHWND = "" Then
    consolewrite("Using active window which is 99% sure equal to the calculator window" & @LF )
    $calcHWND = WinGetHandle("[ACTIVE]")
EndIf

consolewrite($calcHWND)

;Capture the calculator screen
$begin = TimerInit()
winactivate($calcHWND)
Sleep(1000)
_ScreenCapture_CaptureWnd($Bitmap2Filename, $calcHWND,0,0,-1,-1,False)
ConsoleWrite("Saving calculator window " & TimerDiff($begin) & " milliseconds " & @LF)
winmove("[ACTIVE]","",223,341)

;~ Get the base information of the calc window
$aWinPos = WinGetPos($calchWnd)
$aWinCSize = WinGetClientSize($calchWnd)
Sleep(500)
;~ _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
;Capture an area within the calculator area backspace
$begin = TimerInit()
$pos=controlgetpos("","","[CLASS:Button; TEXT:BACKSPACE]")
;~ $pos=controlgetpos("","","[CLASS:Button; INSTANCE:2]")
if @error=1 Then
    consolewrite("Not recognizing so we are on a windows 10 environment most likely, fix by providing manual controls" & @CRLF)
    consolewrite("Read frequently asked questions in Wiki number 31 and use simplespy" & @CRLF)
    local $pos[4]

;~  Backspace button <394;693;75;74>
;~  Button 2         <315;927;75;74>
    consolewrite($awinPos[0] &$awinPos[1] &$awinPos[2] &$awinPos[3])
;~  a 4-element array containing the following information:
;~     $aArray[0] = X position
;~     $aArray[1] = Y position
;~     $aArray[2] = Width
;~     $aArray[3] = Height
;~
;~ Calculate location of backspace button
    winactivate($calcHWND);Make sure calculator is active on the screen
    $pos[0]= (($awinPos[2] / 10) * 8)  ;~ Near 80%
    $pos[1]= (($awinPos[3] / 20) * 7)  ;~ Near 35%
    $pos[2]=65 ;~ Make it a little smaller to only have the icon and not the outside boundaries
    $pos[3]=65 ;~ Make it a little smaller
;~  Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
;~     _UIA_DrawRect($awinPos[0] +$pos[0],$awinPos[0] +$pos[0]+$pos[2]+5,$awinPos[1] +$pos[1],$awinPos[1] +$pos[1]+$pos[2]+5)

;~  Fix quick and dirty as simplespy gives screencoordinates not relative to window
;~     $pos[0]=394 - $aWinpos[0]
;~     $pos[1]=693 - $aWinpos[1]
EndIf


;Q&D calculation of offsets to capture
_ScreenCapture_Capture($Bitmap3Filename, ($aWinPos[0]+$pos[0]) + ($aWinPos[2] - $aWinCSize[0])-3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])-3, $aWinPos[0]+$pos[0]+$pos[2]+3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])+$pos[3]-3,false)
ConsoleWrite("Saving backspacebutton " & TimerDiff($begin) & " milliseconds " & @LF)

; Capture full screen
$begin = TimerInit()
_ScreenCapture_Capture($Bitmap1Filename,0,0,-1,-1,False)
ConsoleWrite("Saving full screen took " & TimerDiff($begin) & " milliseconds " & @LF)
Sleep(500)

func filegetname($f)
    local $i
    $i=stringinstr($f,"\",false,-1)
    if $i > 0 Then
        return stringmid($f,$i+1)
    Else
        return $f
    EndIf
EndFunc

;Do the actual find
Func FindTester($BMP1, $BMP2, $Bool)
    local $tResult
    $start = TimerInit()
    $tResult=findBMP($BMP1,$BMP2, $Bool)
    ConsoleWrite($tResult &  " " & FileGetName($BMP2) & " in " & FileGetName($BMP1) & " ** matchtype " & $Bool & " time elapsed: " & TimerDiff($start) & "  milliseconds" & @LF)
EndFunc

if not $bDebug=True Then
    ; Not very usefull to find full screen and tricky as screen is most likely changed (clock, output from application etc.)
    ; findTester("SCREEN",$Bitmap1Filename,TRUE);Find the full screen itself
    ; findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBFullMatch);Find the full screen itself 31 seconds
    findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBPartialMatch);Find the full screen itself 2.8529088255245  seconds
    ; findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBFullMatch);Find the full screen itself 21 seconds
    findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBPartialMatch);Find the full screen itself 1 seconds

    ; Be aware that overlapping windows, moving things on screen can make it difficult to find on full screen
    findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen
    findTester("SCREEN",$Bitmap2Filename,$c24RGBPArtialMatch);Find the full calculatorscreen with partial match
    findTester("SCREEN",$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen
    findTester("SCREEN",$Bitmap2Filename,$c16RGBPArtialMatch);Find the full calculatorscreen with partial match

    findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen
    findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBPartialMatch);Find the full calculatorscreen with partial match

    findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button
    findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match
    findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button
    findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match

    winactivate($calcHWND);Make sure calculator is active on the screen
    findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button
    findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match
    findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button on active screen
    findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match

    findTester("SCREEN",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button
    findTester("SCREEN",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match
    findTester("SCREEN",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button
    findTester("SCREEN",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match

Else
;~  findTester($Bitmap4Filename,$Bitmap6Filename,$c24RGBFullMatch);Find the cat
;~  findTester($Bitmap1Filename,$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen
    findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen

    winactivate($calcHWND);Make sure calculator is active on the screen
    findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button
EndIf

_GDIPlus_Shutdown()

;** Example end **
#EndRegion

#Region actual functions
;===============================================================================
; Function Name:    findBMP
; Description:    Finds a bitmap (.BMP) in another BMP file (other formats PNG and TIFF work also other formats less usefull GIF, JPEG, Exif, WMF, and EMF should work)
; Syntax:          findBMP($BMP1, $BMP2, $MatchType=TRUE)
;
; Parameter(s):  $BMP1           = Filename of bitmap to search in
;                  $BMP2             = Filename of bitmap to search for
;                  $MatchType       = c24RGBFullMatch, c24RGBPartialMatch, c16RGBFullMatch, c16RGBPartialMatch
;
; Return Value(s):  On Success:   = Returns Array List
;                  On Failure:   = @error 1 (Control was found but there was an error with the DLLCall)
;
; Author(s):        JunkEW
;
; Note(s):
;               * Its never an exact match even with TRUE as last few bits are disposed in algorithm and lines below
;                are not checked under assumption that those are 99.99% of the time correct
;              * locking bits overview http://www.bobpowell.net/lockingbits.htm
; ToDo:
;               * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html
;               * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line)
;               * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen easier)
; Example(s):
;
;===============================================================================

Func findBMP($BMP1, $BMP2, $MatchType=$c24RGBFullMatch)
    Dim $fLine[1];Line number of found line(s), redimmed when second picture size is known
    Dim $BMP1Data="", $BMP1Width=0, $BMP1Height=0, $BMP1LineWidth=0;
    Dim $BMP2Data="", $BMP2Width=0, $BMP2Height=0, $BMP2LineWidth=0
    Dim $foundAt = "", $matchPossible=FALSE, $matchedLines=0, $foundAtLeft=-1, $foundAtTop=-1
    Dim $bestMatchLine=-1, $HighestMatchingLines=-1; For knowing when no match is found where best area is
    Dim $iPos=1;
    dim $imgBytes, $avData;

    local $iFuzzyDist, $searchFor, $iAbove, $bMatchPossible, $aboveLine
    local $j, $imgBits
    local $tDebug

    if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c24RGBPartialMatch) then
        $imgBytes=3
    Else
        $imgBytes=2
    endif

; Load the bitmap to search in
    getImage($BMP1, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes)
    $BMP1Data = BinaryToString($BMP1Data)

; Load the bitmap to find
    getImage($BMP2, $BMP2Data, $BMP2Width, $BMP2Height, $BMP2LineWidth, $imgBytes)
;Make it strings to be able to use string functions for searching
    $BMP2Data = BinaryToString($BMP2Data)

if $bDebug=True Then
;~ START debugging
;Get some debugging information
    FileDelete(@ScriptDir & "\BMP1DATA.TXT")
    FileDelete(@ScriptDir & "\BMP2DATA.TXT")
    FileDelete(@ScriptDir & "\COMPAREDLINES.TXT")

    consolewrite($BMP1Width & @crlf)
    consolewrite($BMP1Height & @crlf)
    consolewrite($BMP1LineWidth & @crlf)
    consolewrite(stringlen($BMP1Data) & @crlf)

    consolewrite(".{" & $BMP1LineWidth & "}"& @CRLF)

    consolewrite(".{" & $BMP2LineWidth & "}"& @CRLF)
    $tDebug=StringRegExpReplace(_stringtohex($BMP1Data),"(.{" & $BMP1LineWidth*2 & "})","$1" & @CRLF)
    consolewrite(@error & @extended & @CRLF)
    FileWrite (@ScriptDir & "\BMP1DATA.TXT" , $tdebug)

    $tDebug=StringRegExpReplace(_stringtohex($BMP2Data),"(.{" & $BMP2LineWidth*2 & "})","$1" & @CRLF)
    consolewrite(@error & @extended & @CRLF)
    FileWrite (@ScriptDir & "\BMP2DATA.TXT" , $tdebug)
;~ END debugging
EndIf

;For reference of line where in BMP2FindIn a line of BMP2Find was found
    If $BMP2Height = 0 Then
        SetError(1,0,0)
        Return  False
    EndIf

    ReDim $fline[$BMP2Height]

;If exact match check every 1 line else do it more fuzzy (as most likely other lines are unique)
    if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c16RGBFullMatch) Then
        $iFuzzyDist = 1
    Else
;Check fuzzy every 10% of lines
        $iFuzzyDist = ceiling(($bmp2height * 0.1))
    endIf

    $begin = TimerInit()
;Look for each line of the bitmap if it exists in the bitmap to find in
;~ Split bitmap to search in lines
;~  $avData=stringregexp($BMP2Data, ".{1," & $BMP2lineWidth & "}+", 3)
dim $searchForRegEx
$searchForRegEx = StringMid($BMP2Data, 1 + (0 * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
$searchForRegEx = $searchForRegEx & ".*" & StringMid($BMP2Data, 1 + ($BMP2Height * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
Local $aArray=stringregexp($BMP1Data,$searchForRegEx,3)
if @error=0 Then
    consolewrite("Regex found " & TimerDiff($begin) & " ms " & @crlf)
;~ For $i = 0 To UBound($aArray) - 1
;~     consolewrite("RegExp Test with Option 3 - " & $i & $aArray[$i])
;~ Next
endif


    For $i = 0 To $BMP2Height - 1
;Minus imgbytes as last bits are padded with unpredictable bytes (24 bits image assumption) or 2 when 16 bits
        $searchFor = StringMid($BMP2Data, 1 + ($i * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
;~      $searchfor=stringleft($avData[$i],$BMP2lineWidth - $imgBytes)

        $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos)
;       $iPos = StringInStr($BMP1Data, $searchFor)


;Look for all lines above if there is also a match
;Not doing it for the lines below as speed is more important and risk of mismatch on lines below is small
        $iAbove=1
        if $iPos > 0 then
            $bMatchPossible=True
            $matchedLines=1;As first found line is matched we start counting
;Location of the match
            $foundAtTop = Int($iPos / $BMP1lineWidth) -$i
            $foundAtLeft =  int(mod($iPos,$bmp1linewidth) / $imgBytes)
        Else
;~          No 1st line found so far nothing to start matching
            $bMatchPossible=false
            exitloop
        endif

        while (($i+$iAbove) <= ($BMP2Height -1)) and ($bMatchPossible=True)
            $searchFor = StringMid($BMP2Data, 1 + (($i + $iAbove) * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes))
;~          $searchfor = stringleft($avData[$i+$iAbove],$BMP2lineWidth - $imgBytes)
            $aboveLine = stringmid($BMP1Data,$iPos + ($iAbove * $BMP1LineWidth), ($BMP2LineWidth - $imgBytes))

            if $bDebug=True Then
            ;~ START debugging
                $tDebug=StringRegExpReplace(_stringtohex($searchfor),"(.{8})","$1_")
                FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf)
                $tDebug=StringRegExpReplace(_stringtohex($aboveline),"(.{8})","$1_")
                FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf)
                If $aboveLine <> $searchFor Then
                    FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "** MISMATCH ** of line " & (1+$iAbove) & " in 2nd bitmap at line " & ($foundAtTop + $i) & @crlf)
                endIf
            ;~ END debugging
            EndIf

            if comparePicLine($aboveline,$searchfor) = false then
                $bMatchPossible=False
;To remember the area with the best match
                if $matchedLines >= $HighestMatchingLines Then
                    $HighestMatchingLines = $matchedLines

    ;Best guess of location
;~                  $foundAtTop = $fline[$i] + $i - $BMP2Height
                    $foundAtTop = Int($iPos / $BMP1lineWidth);+ $i - $BMP2Height
                    $bestMatchLine = Int($iPos / $BMP1lineWidth)
                EndIf
                ExitLoop
            EndIf
            $matchedLines=$matchedLines + 1
            $iAbove=$iAbove+$iFuzzyDist
        WEnd

;If bMatchPossible is still true most likely we have found the bitmap
        if $bmatchPossible = True then
;~          ConsoleWrite("Could match top: " & $foundAtTop & " left: " & $foundAtLeft & " in " & TimerDiff($begin) / 1000 & "  seconds" & @LF)
;           MouseMove($foundatleft,$foundatTop)
            exitloop
        else
;~          consolewrite("i not matched " & $ipos & " " & $matchedlines & @crlf )
        EndIf

    Next

;For some debugging of time
;   if $bMatchPossible = True Then
;       ConsoleWrite("Searching took " & TimerDiff($begin) / 1000 & "  seconds " & @LF)
;   Else
;       ConsoleWrite("NOT FOUND Searching took " & TimerDiff($begin) / 1000 & "  seconds" & @LF)
;   endif

;Return an error if not found else return an array with all information
    if $bMatchPossible = False Then
        SetError(1, 0, 0)
    endif
;   return stringsplit($bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine,";")
    return $bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine
EndFunc;==>findBMP

func comparePicLine($s1,$s2)
    local $sLen, $iMatched
    if $s1=$s2 Then
        return True
    Else
        $iMatched=0
        $sLen=stringlen($s1)
        for $tJ=1 to $slen
            if stringmid($s1,$tJ,1)=stringmid($s2,$tJ,1) Then
                $iMatched=$imatched+1
            Else
                if $bDebug=True Then
                ;~ START debugging
                    FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "Debug mismatch pixel " & $tj & "=" & int($tj/4) & @crlf)
                endif
            endif
        Next
        if  ($iMatched / $sLen ) > $cMatchLinePercentage then
            return true
        Else
            return false
        endif
    EndIf
endfunc

Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3)
local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle

; Load the bitmap to search in
    If $BMPFile="SCREEN" Then
        $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
;try to get a handle
        $handle = WinGetHandle($BMPFile)
        If @error Then
;Assume its an unknown handle so correct filename should be given
            $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)
        Else
            $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False)
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
        EndIf
    EndIf

;Get $tagGDIPBITMAPDATA structure
;~  ConsoleWrite("Bitmap Width:    " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
;~  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison
    if ($imgBytes=2) then
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    Else
        $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
;~      $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB)
    endIf

    If @ERROR Then MsgBox(0,"","Error locking region " & @error)

    $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap.
    $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap.

    $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData")

    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose ($pBitmap)
    _WinAPI_DeleteObject ($pBitmap)

EndFunc;==>GetImage
; Draw rectangle on screen.
Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4)
    Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2
    $x1 = $tLeft
    $x2 = $tRight
    $y1 = $tTop
    $y2 = $tBottom
    $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop)
    $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color)
    $obj_orig = _WinAPI_SelectObject($hDC, $hPen)

    _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right
    _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right
    _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right
    _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left

    ; clear resources
    _WinAPI_SelectObject($hDC, $obj_orig)
    _WinAPI_DeleteObject($hPen)
    _WinAPI_ReleaseDC(0, $hDC)
EndFunc   ;==>_UIA_DrawRect
#EndRegion

 

  • 2 weeks later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...