Search the Community

I wanted to use OpenCV v4+ in AutoIt. I found Opencv UDF on the forum, but there was no support for OpenCV v4+ This UDF provides support for OpenCV v4+ Update There is a new implementation using COM. It is almost as easy as python to use It is also possible to interact with GDI+ Download and extract opencv-4.10.0-windows.exe into a folder Download and extract autoit-opencv-4.10.0-com-v2.6.2.7z into a folder Sources Here Documentation A generated documentation for functions is available here Examples More samples can be found here To run them, please follow these instructions Showing an image #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $img = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\data\lena.jpg")) $cv.imshow("Image", $img) $cv.waitKey() $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Drawing contours #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $img = _OpenCV_imread_and_check("samples\data\pic1.png") Local $img_grey = $cv.cvtColor($img, $CV_COLOR_BGR2GRAY) $cv.threshold($img_grey, 100, 255, $CV_THRESH_BINARY) Local $thresh = $cv.extended[1] Local $contours = $cv.findContours($thresh, $CV_RETR_TREE, $CV_CHAIN_APPROX_SIMPLE) ConsoleWrite("Found " & UBound($contours) & " contours" & @CRLF & @CRLF) $cv.drawContours($img, $contours, -1, _OpenCV_Scalar(0, 0, 255), 2) $cv.imshow("Image", $img) $cv.waitKey() $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Showing an image in autoit GUI #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" #include <GUIConstantsEx.au3> _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return #Region ### START Koda GUI section ### Form= Local $FormGUI = GUICreate("show image in autoit gui", 400, 400, 200, 200) Local $Pic = GUICtrlCreatePic("", 0, 0, 400, 400) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### Local $img = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\data\lena.jpg")) _OpenCV_imshow_ControlPic($img, $FormGUI, $Pic) Local $nMsg While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE ExitLoop EndSwitch WEnd $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Showing an image in an autosized autoit GUI #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" #include <GUIConstantsEx.au3> _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return #Region ### START Koda GUI section ### Form= Local $FormGUI = GUICreate("show image in autoit gui [WINDOW_AUTOSIZE]", 400, 400, 200, 200) Local $Pic = GUICtrlCreatePic("", 0, 0, 400, 400) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### Local $img = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\data\lena.jpg")) ; get the image size and resize the GUI and the PIC control WinMove($FormGUI, "", Default, Default, $img.width, $img.height) GUICtrlSetPos($Pic, Default, Default, $img.width, $img.height) _OpenCV_imshow_ControlPic($img, $FormGUI, $Pic) Local $nMsg While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE ExitLoop EndSwitch WEnd $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Screen capture #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $iLeft = 200 Local $iTop = 200 Local $iWidth = 400 Local $iHeight = 400 Local $aRect[4] = [$iLeft, $iTop, $iWidth, $iHeight] Local $img = _OpenCV_GetDesktopScreenMat($aRect) $cv.imshow("Screen capture", $img) $cv.waitKey() $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Find template #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $img = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\data\mario.png")) Local $tmpl = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\data\mario_coin.png")) ; The higher the value, the higher the match is exact Local $threshold = 0.8 Local $aMatches = _OpenCV_FindTemplate($img, $tmpl, $threshold) Local $aRedColor = _OpenCV_RGB(255, 0, 0) Local $aMatchRect[4] = [0, 0, $tmpl.width, $tmpl.height] For $i = 0 To UBound($aMatches) - 1 $aMatchRect[0] = $aMatches[$i][0] $aMatchRect[1] = $aMatches[$i][1] ; Draw a red rectangle around the matched position $cv.rectangle($img, $aMatchRect, $aRedColor) Next $cv.imshow("Find template example", $img) $cv.waitKey() $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Video capture file #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" #include <Misc.au3> _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $cap = _OpenCV_ObjCreate("cv.VideoCapture").create(_OpenCV_FindFile("samples\data\vtest.avi")) If Not $cap.isOpened() Then ConsoleWriteError("!>Error: cannot open the video file." & @CRLF) Exit EndIf Local $frame = _OpenCV_ObjCreate("cv.Mat") While 1 If _IsPressed("1B") Or _IsPressed(Hex(Asc("Q"))) Then ExitLoop EndIf If Not $cap.read($frame) Then ConsoleWriteError("!>Error: cannot read the video or end of the video." & @CRLF) ExitLoop EndIf $cv.imshow("capture video file", $frame) Sleep(30) WEnd $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Video capture camera #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" #include <Misc.au3> _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $iCamId = 0 Local $cap = _OpenCV_ObjCreate("cv.VideoCapture").create($iCamId) If Not $cap.isOpened() Then ConsoleWriteError("!>Error: cannot open the camera." & @CRLF) Exit EndIf Local $frame = _OpenCV_ObjCreate("cv.Mat") While 1 If _IsPressed("1B") Or _IsPressed(Hex(Asc("Q"))) Then ExitLoop EndIf If $cap.read($frame) Then ;; Flip the image horizontally to give the mirror impression $frame = $cv.flip($frame, 1) $cv.imshow("capture camera", $frame) Else ConsoleWriteError("!>Error: cannot read the camera." & @CRLF) EndIf Sleep(30) WEnd $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _OpenCV_Close() EndFunc ;==>_OnAutoItExit Resize an image with GDI+ #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include "autoit-opencv-com\udf\opencv_udf_utils.au3" _OpenCV_Open("opencv-4.7.0-windows\opencv\build\x64\vc16\bin\opencv_world470.dll", "autoit-opencv-com\autoit_opencv_com470.dll") _GDIPlus_Startup() OnAutoItExitRegister("_OnAutoItExit") Example() Func Example() Local $cv = _OpenCV_get() If Not IsObj($cv) Then Return Local $bMethod = 1 Local $img = _OpenCV_imread_and_check(_OpenCV_FindFile("samples\tutorial_code\yolo\scooter-5180947_1920.jpg")) Local $resized If $bMethod Then $resized = $img.GdiplusResize(600, 400) Else Local $hImage = $img.convertToBitmap() Local $hResizedImage = _GDIPlus_ImageResize($hImage, 600, 400) _GDIPlus_BitmapDispose($hImage) $resized = $cv.createMatFromBitmap($hResizedImage) _GDIPlus_BitmapDispose($hResizedImage) EndIf $cv.imshow("Resized with GDI+", $resized) $cv.waitKey() $cv.destroyAllWindows() EndFunc ;==>Example Func _OnAutoItExit() _GDIPlus_Shutdown() _OpenCV_Close() EndFunc ;==>_OnAutoItExit

I needed a function to automate programs at work that can't be fully automated via Autoits built in functions. For example a virtual machine running on your physical machine, meaning you would need to run an extra script within the virtual machine (if it is even running Windows) in order to automate everything. I came across OpenCV which allows matching/finding a picture in another picture. This would also allow searching for a button/text on the screen in order to press the exact position. Fortunately @mylise already translated all the required OpenCV functions to Autoit, I just had to remove all unnecessary functions to make the script as small as possible. The problem: Using this method, you will never be able to fully automate everything dynamically, as it will only work on the machine with same resolution/dpi settings, same theme etc.. This is only a last resort for programs that can't be automated using the built in Autoit functions. Features: Find a given picture on the entire screen (all monitors) or a certain area on the screen and execute mouse clicks on this position. Adjust the threshold so that the picture doesn't have to match 100%. Debugging options like logging and marking the screen where the picture was found etc. Includes a Snapshot-Tool that creates snapshots of a certain area(buttons, text etc.) on the screen and generates the code that is required for the matching. It can also be used to get the coordinates to a marked area on the screen in order to check only on a certain area for the match picture. Example: Note: The example will probably not work on your computer, depending on the display resolution and dpi settings, as the picture has to match the exact same size on the screen. Please use the included Snapshot-Tool to generate new match pictures and code very easily. #AutoIt3Wrapper_UseX64=n ; In order for the x86 DLLs to work #include "OpenCV-Match_UDF.au3" _OpenCV_Startup();loads opencv DLLs _OpenCV_EnableLogging(True,True,True) ;Logs matches, errors in a log file and autoit console output. ;Please note that these examples might not work as the match pictures have to be found with the exact same size on your screen. ;Example 1 ShellExecute("http://www.tv.com/");Open Website tv.com $Match1 = _MatchPicture(@ScriptDir&"\Match\1.png", 0.70,False,10,500);Try to find the match picture on the screen. Number of tries: 10, Sleep between each try: 500ms. If Not @error Then _MarkMatch($Match1) ;Debugging: Draws a rect on the screen/coordinates of the match to show the user where the match was found Sleep(100) _ClickMouse($Match1, "left",1) ;Calculates the center of the match and clicks the left mouse once on click position EndIf Sleep(1000) ;Example 2, matching on a specific area of the screen ShellExecute("notepad.exe");open nodepad WinWait("[CLASS:Notepad]","",5) WinMove("[CLASS:Notepad]","",0,0,500,500) Local $sCoords[4] = [0, 0, 500,500] $Match2 = _MatchPicture(@ScriptDir&"\Match\2.png", 0.80,$sCoords,3,500) If Not @error Then _MarkMatch($Match2) Sleep(100) _ClickMouse($Match2, "left", 1) EndIf _OpenCV_Shutdown();Closes DLLs So basically, all you need to do is provide a path to the match picture and the function will return you the coordinates (x1,y1,x2,y2) of where the picture has been found on the screen. With these, you can either calculate an exact position for the mouse click or use the "_ClickMouse" function which will execute a mouse click on the center of the coordinates where the picture was found. Credits: @mylise for the OpenCV UDF Download: Includes the required .DLL files of OpenCV. You can also manually download them on the website of OpenCV (Version 3.x doesn't include these anymore, you need to download 2.x). OpenCV_Match.zip

Greetings, here is a Autoit3 v3.3.14.5, 64 Bit installation on windows 10 with current updates. In a simple script I'm Dllcall'ing a 64 bit dll. # T1.au3 Local $name = "T1DLL.dll" if (FileExists($name)) Then ConsoleWrite("DLL '" &$name &"' exists" &@CRLF) EndIf Local $DLL = DllOpen($name) if (@ERROR OR $DLL = -1) then ConsoleWrite("DllOpen ERROR=" &@ERROR &" DLL=" &$DLL &@CRLF) Else ConsoleWrite("DllOpen Success" &@CRLF) endif DllCall($DLL, "none:cdecl", "SomeFunction", "str", "DLL Call from T1.au3") if (@ERROR) then ConsoleWrite("DllCall ERROR=" &@ERROR &" DLL=" &$DLL &@CRLF) Else ConsoleWrite("DllCall Success" &@CRLF) endif The C code contains one 64 bit OpenCV call: cv::destroyAllWindows();. Excuting T1.au3 with SciTE gives: If the c code is compiled *without* the OpenCV call T1.au3 runs successful: I checked T1DLL.dll, it's definitely a 64 bit dll. What could cause the problem? Any hint would be very much appreciated. Thanks Wolf