DepthCharge Posted February 2, 2013 Posted February 2, 2013 Just tested this out and it works great. Little disappointed that it lacks transparency though. I understand it's a work in progress. Hope to see the ToDo list completed. (Would be nicer if the code in the first post was formatted and was in the autoit bbcode just for the sake of readablity)
junkew Posted February 9, 2013 Author Posted February 9, 2013 updated first post for readability FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
JohnOne Posted February 9, 2013 Posted February 9, 2013 Just a heads up junkew. If you toggle edit mode to basic (none of other options available) Your code between [autoit] tags will retain its indenting. AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans.
silly135 Posted April 15, 2013 Posted April 15, 2013 I tried running the example and search however i get the following error --------------------------- AutoIt Error --------------------------- Line 84 (File "C:\Users\RA029440\Desktop\Misc\imagesearch\bmp\BMPexample.au3"): $tResult=findBMP($BMP1,$BMP2, $Bool) $tResult=^ ERROR Error: Unknown function name. --------------------------- OK --------------------------- Does anyone know why? Do i have to name the files from extraction something else? how about the search function itself? and the example file?
JohnOne Posted April 15, 2013 Posted April 15, 2013 UDF and example should be in the same folder. AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans.
xekon Posted July 1, 2013 Posted July 1, 2013 (edited) I tested your function "findBMP()" I saved it to the file "_FindBMP.au3" and then I used the below code to test it, and it works, the items I look for on my screen I have already generated .bmp files for. so I do not need to do that when I run the script, I just need to look for them. expandcollapse popup#RequireAdmin;win7 #include <GDIPlus.au3> #Include <ScreenCapture.au3> #Include <_FindBMP.au3> ;hotkey to run function to find image on screen HotKeySet("{F11}","Fpic") ; If debugging output is wanted to understand and analyse the algorithm put this on true/false Const $bDebug=False Const $cMatchLinePercentage=0.99 ;Minimum matching, put on 1 if exact matching is needed on all lines ;Constants for type of picture matching const $c24RGBFullMatch=1 ;Load as 24 bits and full match const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match const $c16RGBFullMatch=3 ;Load as 16 bits and full match const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match $fscreen = @scriptDir & "\fscreen.bmp" $find1 = @scriptDir & "\item1.bmp" $find2 = @scriptDir & "\item2.bmp" $find3 = @scriptDir & "\item3.bmp" $find4 = @scriptDir & "\item4.bmp" $find5 = @scriptDir & "\item5.bmp" while 1 sleep(1000) wend Func Fpic() _GDIPlus_Startup() _ScreenCapture_Capture($fscreen, 0, 0, -1, -1, False) $tResult = findBMP($fscreen, $find1, $c24RGBFullMatch) MsgBox(4096, "Test", "result: "&$tResult) _GDIPlus_Shutdown() EndFunc I am very interested in getting HBITMAP to work for the full screen capture, because then it does not need to be written to disk which is much faster. I tried what I though would work but it did not: $fscreen = _ScreenCapture_Capture("", 0, 0, -1, -1, False);HBITMAP method does anyone know how to compare a .bmp file to an HBITMAP? solution can be an example or something quick and dirty and I can make it work from there, much appreciated. (just not sure where to look next.) Edited July 1, 2013 by xekon
junkew Posted November 4, 2013 Author Posted November 4, 2013 To find on the screen I checked the library which just is doing If $BMPFile="SCREEN" Then $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap ElseSo you just can do$tResult = findBMP("SCREEN", $find1, $c24RGBFullMatch)No need to first save to disk (I also have doubt if that is really slow nowadays as it will be most likely cached) FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
undefinedspace Posted June 13, 2014 Posted June 13, 2014 First link:http://www.autoitscript.com/forum/index.ph...=pixelchecksum) does not work.
samehvirus Posted August 4, 2015 Posted August 4, 2015 old thread but awesome script . may i ask a question hope junkew still flowing this thread or some one could help me with this code .so this script will help me to detect un object (bmp) on other bmp and it return the coordinate where it founded , this is very good job . but i want it to find all matches when the object exists in image many times not to find it only once thank you in advance .
junkew Posted August 5, 2015 Author Posted August 5, 2015 findBMP still works the way it should and with a loop you can probably find multiple images in one but that was not the intention of the module. speed can be handled with FASM script https://www.autoitscript.com/forum/topic/158264-bmpsearch-search-for-bitmap-within-bitmap-assembly-version/#comment-1173165https://www.autoitscript.com/forum/topic/140478-bitmap-processing-using-raw-code/#comment-987099 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
junkew Posted January 20, 2018 Author Posted January 20, 2018 Checked and updated first post Works with W10 (and earlier) and AutoIt 3.3.14.2 (and earlier) Calculator controls are not recognized so hardcoded that for this example in first post To recognize controls use FAQ 31 this post is about recognizing bitmap in other bitmap No fixes in the algorithm itself it was mainly not working on the calculator as these calc controls are not recognized by AutoIt FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
junkew Posted October 27, 2018 Author Posted October 27, 2018 Works with W10 and AutoIt 3.3.14.5 (Some) Images to be updated for example for finding t on homepage of AutoIt (its a white T nowadays) FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
faustf Posted November 13, 2019 Posted November 13, 2019 hi @junkew i try to use yout program for find in foto other foto but probably i understund nothing or less ,:D i try to simplify your example because is just complicated for me i create this script simply expandcollapse popup#cs ---------------------------------------------------------------------------- AutoIt Version: 3.3.14.5 Author: myName Script Function: Template AutoIt script. #ce ---------------------------------------------------------------------------- ; Script Start - Add your code below here Opt('MustDeclareVars', 1) #include <GDIPlus.au3> #Include <ScreenCapture.au3> #include <string.au3> #include <WinAPI.au3> #include <WindowsConstants.au3> Const $cMatchLinePercentage=0.99 ;Minimum matching, put on 1 if exact matching is needed on all lines ;~ Constants for type of picture matching const $c24RGBFullMatch=1 ;Load as 24 bits and full match const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match const $c16RGBFullMatch=3 ;Load as 16 bits and full match const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match Global $begin,$BitmapData,$pixelFormat Local $OriginalFoto= @ScriptDir & "\source.bmp" Local $FototoSearch= @ScriptDir & "\tofind.bmp" findBMP($OriginalFoto, $FototoSearch, $c24RGBFullMatch) ;=============================================================================== ; Function Name: findBMP ; Description: Finds a bitmap (.BMP) in another BMP file (other formats PNG and TIFF work also other formats less usefull GIF, JPEG, Exif, WMF, and EMF should work) ; Syntax: findBMP($BMP1, $BMP2, $MatchType=TRUE) ; ; Parameter(s): $BMP1 = Filename of bitmap to search in ; $BMP2 = Filename of bitmap to search for ; $MatchType = c24RGBFullMatch, c24RGBPartialMatch, c16RGBFullMatch, c16RGBPartialMatch ; ; Return Value(s): On Success: = Returns Array List ; On Failure: = @error 1 (Control was found but there was an error with the DLLCall) ; ; Author(s): JunkEW ; ; Note(s): ; * Its never an exact match even with TRUE as last few bits are disposed in algorithm and lines below ; are not checked under assumption that those are 99.99% of the time correct ; * locking bits overview http://www.bobpowell.net/lockingbits.htm ; ToDo: ; * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html ; * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line) ; * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen easier) ; Example(s): ; ;=============================================================================== Func findBMP($BMP1, $BMP2, $MatchType=$c24RGBFullMatch) Dim $fLine[1];Line number of found line(s), redimmed when second picture size is known Dim $BMP1Data="", $BMP1Width=0, $BMP1Height=0, $BMP1LineWidth=0; Dim $BMP2Data="", $BMP2Width=0, $BMP2Height=0, $BMP2LineWidth=0 Dim $foundAt = "", $matchPossible=FALSE, $matchedLines=0, $foundAtLeft=-1, $foundAtTop=-1 Dim $bestMatchLine=-1, $HighestMatchingLines=-1; For knowing when no match is found where best area is Dim $iPos=1; dim $imgBytes, $avData; local $iFuzzyDist, $searchFor, $iAbove, $bMatchPossible, $aboveLine local $j, $imgBits local $tDebug if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c24RGBPartialMatch) then $imgBytes=3 Else $imgBytes=2 endif ; Load the bitmap to search in getImage($BMP1, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes) $BMP1Data = BinaryToString($BMP1Data) ; Load the bitmap to find getImage($BMP2, $BMP2Data, $BMP2Width, $BMP2Height, $BMP2LineWidth, $imgBytes) ;Make it strings to be able to use string functions for searching $BMP2Data = BinaryToString($BMP2Data) #cs if $bDebug=True Then ;~ START debugging ;Get some debugging information FileDelete(@ScriptDir & "\BMP1DATA.TXT") FileDelete(@ScriptDir & "\BMP2DATA.TXT") FileDelete(@ScriptDir & "\COMPAREDLINES.TXT") consolewrite($BMP1Width & @crlf) consolewrite($BMP1Height & @crlf) consolewrite($BMP1LineWidth & @crlf) consolewrite(stringlen($BMP1Data) & @crlf) consolewrite(".{" & $BMP1LineWidth & "}"& @CRLF) consolewrite(".{" & $BMP2LineWidth & "}"& @CRLF) $tDebug=StringRegExpReplace(_stringtohex($BMP1Data),"(.{" & $BMP1LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP1DATA.TXT" , $tdebug) $tDebug=StringRegExpReplace(_stringtohex($BMP2Data),"(.{" & $BMP2LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP2DATA.TXT" , $tdebug) ;~ END debugging EndIf #ce ;For reference of line where in BMP2FindIn a line of BMP2Find was found If $BMP2Height = 0 Then SetError(1,0,0) Return False EndIf ReDim $fline[$BMP2Height] ;If exact match check every 1 line else do it more fuzzy (as most likely other lines are unique) if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c16RGBFullMatch) Then $iFuzzyDist = 1 Else ;Check fuzzy every 10% of lines $iFuzzyDist = ceiling(($bmp2height * 0.1)) endIf $begin = TimerInit() ;Look for each line of the bitmap if it exists in the bitmap to find in ;~ Split bitmap to search in lines ;~ $avData=stringregexp($BMP2Data, ".{1," & $BMP2lineWidth & "}+", 3) dim $searchForRegEx $searchForRegEx = StringMid($BMP2Data, 1 + (0 * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) $searchForRegEx = $searchForRegEx & ".*" & StringMid($BMP2Data, 1 + ($BMP2Height * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) Local $aArray=stringregexp($BMP1Data,$searchForRegEx,3) if @error=0 Then consolewrite("Regex found " & TimerDiff($begin) & " ms " & @crlf) ;~ For $i = 0 To UBound($aArray) - 1 ;~ consolewrite("RegExp Test with Option 3 - " & $i & $aArray[$i]) ;~ Next endif For $i = 0 To $BMP2Height - 1 ;Minus imgbytes as last bits are padded with unpredictable bytes (24 bits image assumption) or 2 when 16 bits $searchFor = StringMid($BMP2Data, 1 + ($i * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor=stringleft($avData[$i],$BMP2lineWidth - $imgBytes) $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos) ; $iPos = StringInStr($BMP1Data, $searchFor) ;Look for all lines above if there is also a match ;Not doing it for the lines below as speed is more important and risk of mismatch on lines below is small $iAbove=1 if $iPos > 0 then $bMatchPossible=True $matchedLines=1;As first found line is matched we start counting ;Location of the match $foundAtTop = Int($iPos / $BMP1lineWidth) -$i $foundAtLeft = int(mod($iPos,$bmp1linewidth) / $imgBytes) Else ;~ No 1st line found so far nothing to start matching $bMatchPossible=false exitloop endif while (($i+$iAbove) <= ($BMP2Height -1)) and ($bMatchPossible=True) $searchFor = StringMid($BMP2Data, 1 + (($i + $iAbove) * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor = stringleft($avData[$i+$iAbove],$BMP2lineWidth - $imgBytes) $aboveLine = stringmid($BMP1Data,$iPos + ($iAbove * $BMP1LineWidth), ($BMP2LineWidth - $imgBytes)) #cs if $bDebug=True Then ;~ START debugging $tDebug=StringRegExpReplace(_stringtohex($searchfor),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) $tDebug=StringRegExpReplace(_stringtohex($aboveline),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) If $aboveLine <> $searchFor Then FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "** MISMATCH ** of line " & (1+$iAbove) & " in 2nd bitmap at line " & ($foundAtTop + $i) & @crlf) endIf ;~ END debugging EndIf #ce if comparePicLine($aboveline,$searchfor) = false then $bMatchPossible=False ;To remember the area with the best match if $matchedLines >= $HighestMatchingLines Then $HighestMatchingLines = $matchedLines ;Best guess of location ;~ $foundAtTop = $fline[$i] + $i - $BMP2Height $foundAtTop = Int($iPos / $BMP1lineWidth);+ $i - $BMP2Height $bestMatchLine = Int($iPos / $BMP1lineWidth) EndIf ExitLoop EndIf $matchedLines=$matchedLines + 1 $iAbove=$iAbove+$iFuzzyDist WEnd ;If bMatchPossible is still true most likely we have found the bitmap if $bmatchPossible = True then ;~ ConsoleWrite("Could match top: " & $foundAtTop & " left: " & $foundAtLeft & " in " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; MouseMove($foundatleft,$foundatTop) exitloop else ;~ consolewrite("i not matched " & $ipos & " " & $matchedlines & @crlf ) EndIf Next ;For some debugging of time ; if $bMatchPossible = True Then ; ConsoleWrite("Searching took " & TimerDiff($begin) / 1000 & " seconds " & @LF) ; Else ; ConsoleWrite("NOT FOUND Searching took " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; endif ;Return an error if not found else return an array with all information if $bMatchPossible = False Then SetError(1, 0, 0) endif ; return stringsplit($bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine,";") return $bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine EndFunc;==>findBMP func comparePicLine($s1,$s2) local $sLen, $iMatched if $s1=$s2 Then return True Else $iMatched=0 $sLen=stringlen($s1) for $tJ=1 to $slen if stringmid($s1,$tJ,1)=stringmid($s2,$tJ,1) Then $iMatched=$imatched+1 Else #cs if $bDebug=True Then ;~ START debugging FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "Debug mismatch pixel " & $tj & "=" & int($tj/4) & @crlf) endif #ce endif Next if ($iMatched / $sLen ) > $cMatchLinePercentage then return true Else return false endif EndIf endfunc Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3) local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle ; Load the bitmap to search in If $BMPFile="SCREEN" Then $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap Else ;try to get a handle $handle = WinGetHandle($BMPFile) If @error Then ;Assume its an unknown handle so correct filename should be given $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile) Else $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap EndIf EndIf ;Get $tagGDIPBITMAPDATA structure ;~ ConsoleWrite("Bitmap Width: " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF ) ;~ ConsoleWrite("Bitmap Height: " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF) ;~ 24 bits (3 bytes) or 16 bits (2 bytes) comparison if ($imgBytes=2) then $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) Else $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) endIf If @ERROR Then MsgBox(0,"","Error locking region " & @error) $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up. $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap. $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap. $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap. $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0) $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData") _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData) _GDIPlus_ImageDispose ($pBitmap) _WinAPI_DeleteObject ($pBitmap) EndFunc;==>GetImage ; Draw rectangle on screen. Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2 $x1 = $tLeft $x2 = $tRight $y1 = $tTop $y2 = $tBottom $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop) $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color) $obj_orig = _WinAPI_SelectObject($hDC, $hPen) _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left ; clear resources _WinAPI_SelectObject($hDC, $obj_orig) _WinAPI_DeleteObject($hPen) _WinAPI_ReleaseDC(0, $hDC) EndFunc ;==>_UIA_DrawRect but tell me error looking region 1 2 time o_?_O
junkew Posted November 24, 2019 Author Posted November 24, 2019 your error information is very cryptic. The whole algorithm just tries to find a matching image string in another imagestring and if it finds one it starts to match more lines. If not all lines have a match you get a fuzzy percentage where you can decide on if its match or not match. Its just quicker then doing pixel by pixel compare in AutoIt. FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
HoldanCZ Posted December 14, 2019 Posted December 14, 2019 On 11/24/2019 at 7:24 PM, junkew said: your error information is very cryptic. The whole algorithm just tries to find a matching image string in another imagestring and if it finds one it starts to match more lines. If not all lines have a match you get a fuzzy percentage where you can decide on if its match or not match. Its just quicker then doing pixel by pixel compare in AutoIt. Hey, correct me if I'm wrong please, but fuzz variable doesn't work on the first line of the image u are searching for, so if the first line doesn't have exact match whole script returns False without even checking any more lines right? Is there a way to add the fuzz into the first line of the search as well? $searchFor variable is i believe the first line (that must be exact match) of image you want to search for regardless of what constant for type of picture matching u choose. Otherwise its very neat, easy to use and very fast UDF. Thank you for sharing and updating this. 👍
junkew Posted December 14, 2019 Author Posted December 14, 2019 correct. Whole goal of this function was to find on screen bitmaps not on an pictures or photos. Feel free to experiminet with the algorithm FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
junkew Posted February 17, 2022 Author Posted February 17, 2022 It still works nicely under W10, modified a little so win10 calc example works completely with bitmap searching (but make sure your bitmap matches in detail so it really makes a difference if you have your window active or not have bitmap with red parts on it all small little differences will lead to a mismatch) expandcollapse popup#Region includes Opt('MustDeclareVars', 1) ;#include <GUIConstants.au3> #include <GDIPlus.au3> #Include <ScreenCapture.au3> #include <string.au3> #include <WinAPI.au3> #include <WindowsConstants.au3> #EndRegion ;~ If debugging output is wanted to understand and analyse the algorithm put this on true/false ;~ Const $bDebug=True Const $bDebug=true #region example ;~ Const $cMatchLinePercentage=0.97 ;Minimum matching, put on 1 if exact matching is needed on all lines Const $cMatchLinePercentage=0.99 ;Minimum matching, put on 1 if exact matching is needed on all lines ;~ Constants for type of picture matching const $c24RGBFullMatch=1 ;Load as 24 bits and full match const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match const $c16RGBFullMatch=3 ;Load as 16 bits and full match const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match ; ** Example start ** ; Screen samples Global Const $Bitmap1Filename = @scriptDir & "\FULLSCREEN.BMP" Global Const $Bitmap2Filename = @scriptDir & "\CALCULATOR.BMP" Global Const $Bitmap3Filename = @scriptDir & "\BACKSPACE.BMP" ;BMP samples local $calcHWND, $begin, $pos, $aWinPos, $aWinCSize, $start local $pBitmap, $BitmapData, $pixelFormat Opt("WinTitleMatchMode", 4);Matching windows by advanced options ; Initialize GDI+ library _GDIPlus_Startup () ;Calculator, make sure its started $calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]") If $calcHWND = "" Then Run("calc.exe") Sleep(2000) $calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]") EndIf If $calcHWND = "" Then consolewrite("Using active window which is 99% sure equal to the calculator window" & @LF ) $calcHWND = WinGetHandle("[ACTIVE]") EndIf consolewrite($calcHWND) ;Capture the calculator screen $begin = TimerInit() winactivate($calcHWND) Sleep(1000) _ScreenCapture_CaptureWnd($Bitmap2Filename, $calcHWND,0,0,-1,-1,False) ConsoleWrite("Saving calculator window " & TimerDiff($begin) & " milliseconds " & @LF) winmove("[ACTIVE]","",223,341) ;~ Get the base information of the calc window $aWinPos = WinGetPos($calchWnd) $aWinCSize = WinGetClientSize($calchWnd) Sleep(500) ;~ _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) ;Capture an area within the calculator area backspace $begin = TimerInit() $pos=controlgetpos("","","[CLASS:Button; TEXT:BACKSPACE]") ;~ $pos=controlgetpos("","","[CLASS:Button; INSTANCE:2]") if @error=1 Then consolewrite("Not recognizing so we are on a windows 10 environment most likely, fix by providing manual controls" & @CRLF) consolewrite("Read frequently asked questions in Wiki number 31 and use simplespy" & @CRLF) local $pos[4] ;~ Backspace button <394;693;75;74> ;~ Button 2 <315;927;75;74> consolewrite($awinPos[0] &$awinPos[1] &$awinPos[2] &$awinPos[3]) ;~ a 4-element array containing the following information: ;~ $aArray[0] = X position ;~ $aArray[1] = Y position ;~ $aArray[2] = Width ;~ $aArray[3] = Height ;~ ;~ Calculate location of backspace button winactivate($calcHWND);Make sure calculator is active on the screen $pos[0]= (($awinPos[2] / 10) * 8) ;~ Near 80% $pos[1]= (($awinPos[3] / 20) * 7) ;~ Near 35% $pos[2]=65 ;~ Make it a little smaller to only have the icon and not the outside boundaries $pos[3]=65 ;~ Make it a little smaller ;~ Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) ;~ _UIA_DrawRect($awinPos[0] +$pos[0],$awinPos[0] +$pos[0]+$pos[2]+5,$awinPos[1] +$pos[1],$awinPos[1] +$pos[1]+$pos[2]+5) ;~ Fix quick and dirty as simplespy gives screencoordinates not relative to window ;~ $pos[0]=394 - $aWinpos[0] ;~ $pos[1]=693 - $aWinpos[1] EndIf ;Q&D calculation of offsets to capture _ScreenCapture_Capture($Bitmap3Filename, ($aWinPos[0]+$pos[0]) + ($aWinPos[2] - $aWinCSize[0])-3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])-3, $aWinPos[0]+$pos[0]+$pos[2]+3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])+$pos[3]-3,false) ConsoleWrite("Saving backspacebutton " & TimerDiff($begin) & " milliseconds " & @LF) ; Capture full screen $begin = TimerInit() _ScreenCapture_Capture($Bitmap1Filename,0,0,-1,-1,False) ConsoleWrite("Saving full screen took " & TimerDiff($begin) & " milliseconds " & @LF) Sleep(500) func filegetname($f) local $i $i=stringinstr($f,"\",false,-1) if $i > 0 Then return stringmid($f,$i+1) Else return $f EndIf EndFunc ;Do the actual find Func FindTester($BMP1, $BMP2, $Bool) local $tResult $start = TimerInit() $tResult=findBMP($BMP1,$BMP2, $Bool) ConsoleWrite($tResult & " " & FileGetName($BMP2) & " in " & FileGetName($BMP1) & " ** matchtype " & $Bool & " time elapsed: " & TimerDiff($start) & " milliseconds" & @LF) EndFunc if not $bDebug=True Then ; Not very usefull to find full screen and tricky as screen is most likely changed (clock, output from application etc.) ; findTester("SCREEN",$Bitmap1Filename,TRUE);Find the full screen itself ; findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBFullMatch);Find the full screen itself 31 seconds findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBPartialMatch);Find the full screen itself 2.8529088255245 seconds ; findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBFullMatch);Find the full screen itself 21 seconds findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBPartialMatch);Find the full screen itself 1 seconds ; Be aware that overlapping windows, moving things on screen can make it difficult to find on full screen findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c24RGBPArtialMatch);Find the full calculatorscreen with partial match findTester("SCREEN",$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c16RGBPArtialMatch);Find the full calculatorscreen with partial match findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBPartialMatch);Find the full calculatorscreen with partial match findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match winactivate($calcHWND);Make sure calculator is active on the screen findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button on active screen findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match findTester("SCREEN",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester("SCREEN",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester("SCREEN",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button findTester("SCREEN",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match Else ;~ findTester($Bitmap4Filename,$Bitmap6Filename,$c24RGBFullMatch);Find the cat ;~ findTester($Bitmap1Filename,$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen winactivate($calcHWND);Make sure calculator is active on the screen findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button EndIf _GDIPlus_Shutdown() ;** Example end ** #EndRegion #Region actual functions ;=============================================================================== ; Function Name: findBMP ; Description: Finds a bitmap (.BMP) in another BMP file (other formats PNG and TIFF work also other formats less usefull GIF, JPEG, Exif, WMF, and EMF should work) ; Syntax: findBMP($BMP1, $BMP2, $MatchType=TRUE) ; ; Parameter(s): $BMP1 = Filename of bitmap to search in ; $BMP2 = Filename of bitmap to search for ; $MatchType = c24RGBFullMatch, c24RGBPartialMatch, c16RGBFullMatch, c16RGBPartialMatch ; ; Return Value(s): On Success: = Returns Array List ; On Failure: = @error 1 (Control was found but there was an error with the DLLCall) ; ; Author(s): JunkEW ; ; Note(s): ; * Its never an exact match even with TRUE as last few bits are disposed in algorithm and lines below ; are not checked under assumption that those are 99.99% of the time correct ; * locking bits overview http://www.bobpowell.net/lockingbits.htm ; ToDo: ; * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html ; * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line) ; * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen easier) ; Example(s): ; ;=============================================================================== Func findBMP($BMP1, $BMP2, $MatchType=$c24RGBFullMatch) Dim $fLine[1];Line number of found line(s), redimmed when second picture size is known Dim $BMP1Data="", $BMP1Width=0, $BMP1Height=0, $BMP1LineWidth=0; Dim $BMP2Data="", $BMP2Width=0, $BMP2Height=0, $BMP2LineWidth=0 Dim $foundAt = "", $matchPossible=FALSE, $matchedLines=0, $foundAtLeft=-1, $foundAtTop=-1 Dim $bestMatchLine=-1, $HighestMatchingLines=-1; For knowing when no match is found where best area is Dim $iPos=1; dim $imgBytes, $avData; local $iFuzzyDist, $searchFor, $iAbove, $bMatchPossible, $aboveLine local $j, $imgBits local $tDebug if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c24RGBPartialMatch) then $imgBytes=3 Else $imgBytes=2 endif ; Load the bitmap to search in getImage($BMP1, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes) $BMP1Data = BinaryToString($BMP1Data) ; Load the bitmap to find getImage($BMP2, $BMP2Data, $BMP2Width, $BMP2Height, $BMP2LineWidth, $imgBytes) ;Make it strings to be able to use string functions for searching $BMP2Data = BinaryToString($BMP2Data) if $bDebug=True Then ;~ START debugging ;Get some debugging information FileDelete(@ScriptDir & "\BMP1DATA.TXT") FileDelete(@ScriptDir & "\BMP2DATA.TXT") FileDelete(@ScriptDir & "\COMPAREDLINES.TXT") consolewrite($BMP1Width & @crlf) consolewrite($BMP1Height & @crlf) consolewrite($BMP1LineWidth & @crlf) consolewrite(stringlen($BMP1Data) & @crlf) consolewrite(".{" & $BMP1LineWidth & "}"& @CRLF) consolewrite(".{" & $BMP2LineWidth & "}"& @CRLF) $tDebug=StringRegExpReplace(_stringtohex($BMP1Data),"(.{" & $BMP1LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP1DATA.TXT" , $tdebug) $tDebug=StringRegExpReplace(_stringtohex($BMP2Data),"(.{" & $BMP2LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP2DATA.TXT" , $tdebug) ;~ END debugging EndIf ;For reference of line where in BMP2FindIn a line of BMP2Find was found If $BMP2Height = 0 Then SetError(1,0,0) Return False EndIf ReDim $fline[$BMP2Height] ;If exact match check every 1 line else do it more fuzzy (as most likely other lines are unique) if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c16RGBFullMatch) Then $iFuzzyDist = 1 Else ;Check fuzzy every 10% of lines $iFuzzyDist = ceiling(($bmp2height * 0.1)) endIf $begin = TimerInit() ;Look for each line of the bitmap if it exists in the bitmap to find in ;~ Split bitmap to search in lines ;~ $avData=stringregexp($BMP2Data, ".{1," & $BMP2lineWidth & "}+", 3) dim $searchForRegEx $searchForRegEx = StringMid($BMP2Data, 1 + (0 * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) $searchForRegEx = $searchForRegEx & ".*" & StringMid($BMP2Data, 1 + ($BMP2Height * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) Local $aArray=stringregexp($BMP1Data,$searchForRegEx,3) if @error=0 Then consolewrite("Regex found " & TimerDiff($begin) & " ms " & @crlf) ;~ For $i = 0 To UBound($aArray) - 1 ;~ consolewrite("RegExp Test with Option 3 - " & $i & $aArray[$i]) ;~ Next endif For $i = 0 To $BMP2Height - 1 ;Minus imgbytes as last bits are padded with unpredictable bytes (24 bits image assumption) or 2 when 16 bits $searchFor = StringMid($BMP2Data, 1 + ($i * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor=stringleft($avData[$i],$BMP2lineWidth - $imgBytes) $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos) ; $iPos = StringInStr($BMP1Data, $searchFor) ;Look for all lines above if there is also a match ;Not doing it for the lines below as speed is more important and risk of mismatch on lines below is small $iAbove=1 if $iPos > 0 then $bMatchPossible=True $matchedLines=1;As first found line is matched we start counting ;Location of the match $foundAtTop = Int($iPos / $BMP1lineWidth) -$i $foundAtLeft = int(mod($iPos,$bmp1linewidth) / $imgBytes) Else ;~ No 1st line found so far nothing to start matching $bMatchPossible=false exitloop endif while (($i+$iAbove) <= ($BMP2Height -1)) and ($bMatchPossible=True) $searchFor = StringMid($BMP2Data, 1 + (($i + $iAbove) * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor = stringleft($avData[$i+$iAbove],$BMP2lineWidth - $imgBytes) $aboveLine = stringmid($BMP1Data,$iPos + ($iAbove * $BMP1LineWidth), ($BMP2LineWidth - $imgBytes)) if $bDebug=True Then ;~ START debugging $tDebug=StringRegExpReplace(_stringtohex($searchfor),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) $tDebug=StringRegExpReplace(_stringtohex($aboveline),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) If $aboveLine <> $searchFor Then FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "** MISMATCH ** of line " & (1+$iAbove) & " in 2nd bitmap at line " & ($foundAtTop + $i) & @crlf) endIf ;~ END debugging EndIf if comparePicLine($aboveline,$searchfor) = false then $bMatchPossible=False ;To remember the area with the best match if $matchedLines >= $HighestMatchingLines Then $HighestMatchingLines = $matchedLines ;Best guess of location ;~ $foundAtTop = $fline[$i] + $i - $BMP2Height $foundAtTop = Int($iPos / $BMP1lineWidth);+ $i - $BMP2Height $bestMatchLine = Int($iPos / $BMP1lineWidth) EndIf ExitLoop EndIf $matchedLines=$matchedLines + 1 $iAbove=$iAbove+$iFuzzyDist WEnd ;If bMatchPossible is still true most likely we have found the bitmap if $bmatchPossible = True then ;~ ConsoleWrite("Could match top: " & $foundAtTop & " left: " & $foundAtLeft & " in " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; MouseMove($foundatleft,$foundatTop) exitloop else ;~ consolewrite("i not matched " & $ipos & " " & $matchedlines & @crlf ) EndIf Next ;For some debugging of time ; if $bMatchPossible = True Then ; ConsoleWrite("Searching took " & TimerDiff($begin) / 1000 & " seconds " & @LF) ; Else ; ConsoleWrite("NOT FOUND Searching took " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; endif ;Return an error if not found else return an array with all information if $bMatchPossible = False Then SetError(1, 0, 0) endif ; return stringsplit($bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine,";") return $bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine EndFunc;==>findBMP func comparePicLine($s1,$s2) local $sLen, $iMatched if $s1=$s2 Then return True Else $iMatched=0 $sLen=stringlen($s1) for $tJ=1 to $slen if stringmid($s1,$tJ,1)=stringmid($s2,$tJ,1) Then $iMatched=$imatched+1 Else if $bDebug=True Then ;~ START debugging FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "Debug mismatch pixel " & $tj & "=" & int($tj/4) & @crlf) endif endif Next if ($iMatched / $sLen ) > $cMatchLinePercentage then return true Else return false endif EndIf endfunc Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3) local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle ; Load the bitmap to search in If $BMPFile="SCREEN" Then $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap Else ;try to get a handle $handle = WinGetHandle($BMPFile) If @error Then ;Assume its an unknown handle so correct filename should be given $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile) Else $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap EndIf EndIf ;Get $tagGDIPBITMAPDATA structure ;~ ConsoleWrite("Bitmap Width: " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF ) ;~ ConsoleWrite("Bitmap Height: " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF) ;~ 24 bits (3 bytes) or 16 bits (2 bytes) comparison if ($imgBytes=2) then $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) Else $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) endIf If @ERROR Then MsgBox(0,"","Error locking region " & @error) $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up. $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap. $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap. $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap. $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0) $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData") _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData) _GDIPlus_ImageDispose ($pBitmap) _WinAPI_DeleteObject ($pBitmap) EndFunc;==>GetImage ; Draw rectangle on screen. Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2 $x1 = $tLeft $x2 = $tRight $y1 = $tTop $y2 = $tBottom $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop) $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color) $obj_orig = _WinAPI_SelectObject($hDC, $hPen) _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left ; clear resources _WinAPI_SelectObject($hDC, $obj_orig) _WinAPI_DeleteObject($hPen) _WinAPI_ReleaseDC(0, $hDC) EndFunc ;==>_UIA_DrawRect #EndRegion FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
skynet0504 Posted March 2, 2022 Posted March 2, 2022 34 minutes ago, skynet0504 said: I'm "Error locking region 10" What Happened I'm Get it , URL Wrong
junkew Posted March 2, 2022 Author Posted March 2, 2022 region locking problems most likely to issues with GDI itself. Check first if this works https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_Capture.htm FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now