Leaderboard
Popular Content
Showing content with the highest reputation on 11/01/2019 in all areas
-
Version 2025.5.25.5
11,519 downloads
High-Performance ImageSearch UDF & DLL for AutoIt This project provides a highly optimized UDF (User Defined Function) and two versions of a DLL (Dynamic-Link Library) for fast and flexible image searching on the screen using AutoIt. It serves as a powerful replacement for standard image search functions, delivering superior speed, especially on modern CPUs, by leveraging advanced SIMD instructions. ✨ Key Features Superior Speed: The modern version utilizes the AVX2 instruction set to accelerate the search speed by several factors compared to traditional methods. Two DLL Versions: Provides both a modern version (optimized for speed) and a legacy version (for Windows XP support). Multi-Image Search: Search for multiple image files in a single function call by separating paths with a pipe (|). Multi-Scale Searching: Automatically search for an image across a range of sizes (e.g., from 80% to 120% of its original size). Color Tolerance: Find images even with slight color variations by setting a tolerance value (0-255). Transparent Color Support: Specify a color in the source image to be ignored during the search. Flexible Result Handling: Find and return the first match. Find and return all matches on the screen. Limit the maximum number of results. Smart (Hybrid) DLL Loading: The UDF prioritizes an external DLL for maximum performance and automatically falls back to an embedded DLL to ensure the script always runs. Unicode Support: Works flawlessly with file paths containing Unicode characters. Thread-Safe: The DLL is designed to operate stably in multi-threaded scenarios. Debug Information: Provides an option to return a detailed debug string for easy troubleshooting. 🚀 The Two DLL Versions The project offers two DLL versions to meet different needs: 1. ImageSearch_x86.dll ImageSearch_x64.dll (Modern Version) (Attached in the same UDF folder - Because the DLL file with AVX2 support is large in size) This is the recommended version for most users. Strengths: AVX2 Support: Leverages Advanced Vector Extensions 2 on modern CPUs to process multiple pixels in parallel, resulting in extremely fast search speeds. Built with modern C++, ensuring stability and efficiency. Limitations: Not compatible with Windows XP. When to use: When you need maximum performance on Windows 7, 8, 10, 11, and newer. 2. ImageSearch_XP.dll (Legacy Version) (Embedded in UDF code) This version is created for backward compatibility. Strengths: Windows XP Compatibility: Works well on the Windows XP (SP3) operating system. Limitations: No AVX2 Support: Search speed will be significantly slower than the modern version on AVX2-supported CPUs. When to use: When your script must run in a Windows XP environment. ⚙️ How the UDF Works The ImageSearch_UDF.au3 file uses a very smart "hybrid" DLL loading mechanism: Prioritize External DLL: When the _ImageSearch function is called, the UDF first looks for ImageSearch_x86.dll and ImageSearch_x64.dll in the same directory as the script (@ScriptDir). If found, it uses this file to achieve the best performance (with AVX2 if available). Fallback to Embedded DLL: If the external DLL is not found, the UDF will automatically extract and use a legacy (non-AVX2) compatible DLL version that is embedded within it as a hex string. ➡️ This ensures that your script can always run, even if you forget to copy the DLL file. However, for the highest speed, always place the modern ImageSearch_x86.dll and ImageSearch_x64.dll next to your script. 📦 Setup Place the DLL file: Copy ImageSearch_x86.dll and ImageSearch_x64.dll (the modern version) into the same directory as your AutoIt script file. Include the UDF in your script: Use the line #include <ImageSearch_UDF.au3> in your script. 📖 API Reference The main function for performing an image search. _ImageSearch($sImageFile, [$iLeft = 0], [$iTop = 0], [$iRight = 0], [$iBottom = 0], [$iTolerance = 10], [$iTransparent = 0xFFFFFFFF], [$iMultiResults = 0], [$iCenterPOS = 1], [$iReturnDebug = 0], [$fMinScale = 1.0], [$fMaxScale = 1.0], [$fScaleStep = 0.1], [$iFindAllOccurrences = 0]) Parameters Parameter Type Default Description $sImageFile String - Path to the image file. To search for multiple images, separate paths with a pipe (` $iLeft Int 0 The left coordinate of the search area. 0 defaults to the entire screen. $iTop Int 0 The top coordinate of the search area. 0 defaults to the entire screen. $iRight Int 0 The right coordinate of the search area. 0 defaults to the entire screen. $iBottom Int 0 The bottom coordinate of the search area. 0 defaults to the entire screen. $iTolerance Int 10 Color tolerance (0-255). A higher value allows for greater color variation. $iTransparent Int 0xFFFFFFFF The color (in 0xRRGGBB format) to be ignored in the source image. 0xFFFFFFFF means no transparency. $iMultiResults Int 0 The maximum number of results to return. 0 means no limit. $iCenterPOS Bool 1 (True) If True, the returned X/Y coordinates will be the center of the found image. If False, they will be the top-left corner. $iReturnDebug Bool 0 (False) If True, the function returns a debug string instead of the results array. $fMinScale Float 1.0 The minimum scaling factor for the search (e.g., 0.8 for 80%). Must be >= 0.1. $fMaxScale Float 1.0 The maximum scaling factor for the search (e.g., 1.2 for 120%). $fScaleStep Float 0.1 The increment to use when searching between min and max scales. Must be >= 0.01. $iFindAllOccurrences Bool 0 (False) If False, the search stops after the first match. If True, it finds all possible matches. Return Value On Success: Returns a 2D array containing the coordinates of the found images. $aResult[0][0] = The number of matches found. $aResult[1] to $aResult[$aResult[0][0]] = An array for each match. $aResult[$i][0] = X coordinate $aResult[$i][1] = Y coordinate $aResult[$i][2] = Width of the found image $aResult[$i][3] = Height of the found image On Failure / No Match: Sets @error to 1 and returns 0. In Debug Mode: If $iReturnDebug is True, returns a string containing detailed information about the last search operation. 💻 Examples Example 1: Basic Search Find the first occurrence of button.png on the screen. #include <ImageSearch_UDF.au3> Local $aResult = _ImageSearch("C:\images\button.png") If @error Then MsgBox(48, "Error", "Image not found on screen.") Else Local $iCount = $aResult[0][0] Local $iX = $aResult[1][0] Local $iY = $aResult[1][1] MsgBox(64, "Success", "Found " & $iCount & " image(s). First match is at: " & $iX & ", " & $iY) MouseMove($iX, $iY, 20) ; Move mouse to the center of the found image EndIf Example 2: Advanced Search (Multiple Images, Tolerance, Scaling) Search for icon1.png or icon2.png within a specific region, with a tolerance of 20 and scaling from 90% to 110%. Find all occurrences. #include <ImageSearch_UDF.au3> Local $sImages = "icon1.png|icon2.png" Local $iTolerance = 20 Local $fMinScale = 0.9 Local $fMaxScale = 1.1 Local $fStep = 0.05 Local $aResult = _ImageSearch($sImages, 500, 300, 1200, 800, $iTolerance, 0xFFFFFFFF, 0, True, False, $fMinScale, $fMaxScale, $fStep, True) If @error Then MsgBox(48, "Error", "No matching images found in the specified region.") Else Local $iCount = $aResult[0][0] ConsoleWrite("Found " & $iCount & " total matches." & @CRLF) For $i = 1 To $iCount ConsoleWrite("Match #" & $i & ": X=" & $aResult[$i][0] & ", Y=" & $aResult[$i][1] & ", W=" & $aResult[$i][2] & ", H=" & $aResult[$i][3] & @CRLF) Next EndIf Example 3: Using Debug Mode To diagnose issues, use the $iReturnDebug parameter. #include <ImageSearch_UDF.au3> Local $2dDLLResult = _ImageSearch("image_not_exist.png", 0, 0, 0, 0, 10, 0xFFFFFFFF, 0, True, True) ConsoleWrite(">> DLL Return: " & $g_sLastDllReturn & @CRLF) ; Example output: {0}[No Match Found] | DEBUG: File=image_not_exist.png, Rect=(0,0,1920,1080), Tol=10, Trans=0xffffffff, Multi=0, Center=1, FindAll=0, AVX2=true, Scale=(1.00,1.00,0.10) SPECIAL NOTE: The function always returns a 2D array for both results and errors. Credits Author: Dao Van Trong - TRONG.PRO Source Dll on GitHub: daovantrong/ImageSearchDLL: A DLL for finding an image on the screen1 point -
BugFix version - 27 Dec 23 Fixed: No default value set for the MaxWidth parameter - took 12 years for someone to notice it! New UDF and new examples below and in zip. I just realised that although I have posted versions of this UDF many times in Help topics, I had never actually posted a "final" version here in Example scripts - better late than never, I suppose! StringSize takes a text string and calculates the size of label required to hold it as well as formatting the string to fit. Now AutoIt will, of course, size a label automatically to fit a text string, but it will not format the string in any way - what you use as the string is what you get in the label. If you do set any label sizes the text will be wrapped, but you can only determine the correct size of the label by trial and error. StringSize will, however, reformat the string to fit in a given width and tell you the required height so that you can read it all - whatever the font type or size - and you do not have to do the formatting beforehand. Here is a simple example to show what I mean (you need the UDF in the same folder for all the examples): And here is an example showing how StringSize can deal with different fonts and text sizes: You can see that the GUI is perfectly sized each time and that the button is always the right size and in the right place. StringSize returns an array which contains the formatted text to display and the size of the label needed to display it. All you need to do is to use the array elements when you create your label. Try changing the values in $aFont and $aSize if you want to try you own favourites - but beware, StringSize will return an error if you make the size so large that it cannot fit a word into the label width. NEW A more complex example showing how formatted and unformatted text can be sized correctly. The width of GUI holding the unformatted text varies randomly in width (the current value is displayed at top right): NEW And a final example showing how you can get your text in the largest possible font to fit in a given space: Finally here is the UDF itself: And all 5 files in zip format: StringSize.zip I hope you find this useful - I certainly do. M231 point
-
Okay, now we are getting somewhere. Since it appears Selenium has a COM interface, I guess the next question is whether you want to stick with Selenium for your browser automation or whether you want to do all of the automation (browser & application) using just AutoIt. AutoIt has the ability to communicate with COM interfaces that utilize the iDispatch interface. That means that you may be able to use Selenium in AutoIt instead of AutoIt in Excel. Since this is an AutoIt forum, I think you can guess which way might get you the most help here on this forum. I must admit that this is a rather ambitious project for someone who is, admittedly, "very new to programming". However, if you are truly interested in learning AutoIt, it could be a very good one to start with since it will expose you to many different subjects (browser automation, app automation, file handling, general coding principles like looping, string handling, boolean logic, etc).1 point
-
Get the name of the last called function
seadoggie01 reacted to Gianni for a topic
yes, right first function call "freeze" the "first level" function name until you will 'free' it by calling the _functionname() function (without parameters) P.S. I don't know if I understood myself ... p.p.s opsss, @water posted the above post just a bit before mine (I hadn't time to read it)1 point -
WebDriver UDF - Help & Support
Danp2 reacted to Davidowicza for a topic
@Danp2 Thank you again! I figured I had to be missing something obvious... Spent about 4 hours trying to figure it out but never guessed it was because of how I was entering the iFrame...it works like a charm!1 point -
WebDriver UDF - Help & Support
Davidowicza reacted to Danp2 for a topic
@Davidowicza You aren't correctly entering the frame. Please review the DemoFrames function in wd_demo.au3 for a working example. P.S. Currently the _WD_Frame* functions don't set the @error flag like most of the other functions in the UDF.1 point -
Error using Autoit through VBA
seadoggie01 reacted to TheXman for a topic
I would strongly suggest that you use the "timeout" parameter with WinWaitActive(). Without it, you could easily get into a situation where your script hangs indefinitely because the specified title doesn't exist. For example, if you made a typo in the title name or the application changed what you expected the title to be, then your script will hang until it is killed. A simple example: Sub test() Dim oAutoIt As AutoItX3 Set oAutoIt = New AutoItX3 'Run Notepad oAutoIt.Run "Notepad.exe", "", oAutoIt.SW_SHOW 'Make sure that it is active and send keys to it oAutoIt.WinActivate "Untitled - Notepad" If oAutoIt.WinWaitActive("Untitled - Notepad", "", 3) Then oAutoIt.Send "Hello World{!}" oAutoIt.Send "^p" End If End Sub1 point -
Two Arrays within loop
FrancescoDiMuro reacted to Subz for a topic
Wouldn't it be easier just to open them all at once, for example: #include <Array.au3> Global $g_aSiteUrls[4] $g_aSiteUrls[0] = "https://www.youtube.com" $g_aSiteUrls[1] = "https://www.amazon.com" $g_aSiteUrls[2] = "https://www.nytimes.com" $g_aSiteUrls[3] = "https://www.nike.com" Global $g_sGoogleChrome = RegRead("HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\Chrome.exe", "") If FileExists($g_sGoogleChrome) = 0 Then Exit Run($g_sGoogleChrome & " " & _ArrayToString($g_aSiteUrls, " "))1 point -
Custom Gui box with a button
seadoggie01 reacted to JLogan3o13 for a topic
I don't think this is anything we want to support.1 point -
Or you can directly use curl.exe instead of running it from cmd.exe curl available https://curl.haxx.se/windows/ Or you can use the curl udf available1 point
-
Image not found
Polistotele reacted to Jos for a topic
Have you checked what statement is on line 44 of the Imagesearch.au3 include you use? Jos1 point -
Image not found
Polistotele reacted to FrancescoDiMuro for a topic
@Polistotele What application are you trying to automate?1 point -
Image not found
Polistotele reacted to Jos for a topic
Sorry to be a pain, but just stick to the one topic I already moved for you. Jos1 point -
Automating Notepad This example is a response to a request by mLipok for a simple Notepad automation. Task: Create a simple script to fill up "Edit1" with HelloWorld, click "Save As...", enter filename and finally save it to disc. This involves automating Notepad or more generally an Edit control, it involves automating a menu and a Save As dialog. Note that "Edit1" (from AutoIt Window Info tool) is not a valid control identification in UI Automation code. Do not open Notepad in advance. This will be done in the code. Start the code by opening Notepad and create the UI Automation object through IUIAutomation interface. I almost always create the Desktop element. Note how you first get a pointer to the element (GetRootElement method of $oUIAutomation object), and then creates the object that represents the element through IUIAutomationElement interface. UI Automation elements are nearly always created this way through a pointer. #include "..\..\Includes\CUIAutomation2.au3" Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc Open Examples\Notepad\Notepad.au3 (empty) in SciTE. Add the code, run the code with F5 and leave Notepad open. Check that SciTE output is OK. Next step is to get the Notepad window as an UI Automation element. Open UIASpy (included in zip-file, keep UIASpy open throughout the entire example), place mouse cursor over Notepad title bar and press F2: (You can delete other windows in UIASpy by right-clicking treeview top windows (level one items).) $UIA_ClassNamePropertyId = "Notepad" is used to create a property condition to identify Notepad. Note that you can copy this line in UIASpy listview: Select the line in the listview, click "Right pane" item in main menu, click "Copy all or selected items to clipboard", paste the line into your code editor. Add this code to the existing code: ; --- Notepad window --- ConsoleWrite( "--- Notepad window ---" & @CRLF ) Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) EndFunc Note again how we first get a pointer to Notepad (FindFirst method of $oDesktop object, $oDesktop because the Desktop is the parent element of Notepad) and then creates the object. Also note that $oDesktop and $oNotepad are created in exactly the same way. Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. Now, the actual automation starts: Fill up "Edit1" with HelloWorld. An automation task is almost always done in two or three steps: 1. Create condition to find element 2. Find the UI Automation element 3. Extract info or perform action Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included). To get the Notepad window above we only used two steps. The first automation task is to perform an action to fill out the Edit control with "HelloWorld". Place mouse cursor in the Edit control and press F1: (If there are two or more Notepad windows in UIASpy, you can delete the old windows (red) by right-clicking the treeview.) This time we can use $UIA_AutomationIdPropertyId = "15" to create a property condition to identify the Edit control. Add this code: ; --- Fill Edit element --- ConsoleWrite( "--- Fill Edit element ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) EndFunc $oNotepad.FindFirst means that the search for the Edit control starts in the Notepad window. $oEdit reprecents the Edit control. Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. The Edit control must be filled with "HelloWorld". This is an action that has to be performed on the Edit control. The following actions are available: (Scroll down UIASpy listview to see Control Patterns (actions).) The interesting action or pattern here is the ValuePattern. The action is created and performed this way: Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) EndFunc $oEdit.GetCurrentPattern means that a pattern is created that performs actions on the Edit control. GetCurrentPattern method returns a pointer where from the ValuePattern object ($oValue) can be created. $UIA_ValuePatternId and the ValuePattern interface is defined in CUIAutomation2.au3. The $oValue object represents the ValuePattern interface for the Edit control. $oValue.SetValue( "HelloWorld" ) sets the text in the Edit control. An action or pattern is almost always created and performed this way. Add the code, close the previous Notepad, run the code and leave Notepad open. Open File menu. Place mouse cursor over the File menu and press F1: (Scroll down UIASpy listview to see Control Patterns (actions) in bottom of image.) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) EndFunc The Invoke action or pattern is used to click and open the File menu. Note the similarity between the code for InvokePattern and ValuePattern above. When a menu, dialog box or child window is opened it need some time to open. You must add a Sleep statement. Add the code, change "File" to the corresponding word in your own language (copy the word from the listview), close the previous Notepad (don't save), run the code and leave Notepad open. Click "Save As..." menu. If the File menu that was opened above is closed, then open it manually with a mouse click. Place mouse cursor over the "Save As..." menu and press F1: ; --- Click "Save As..." menu --- ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF ) ; Reuse $pCondition1 above ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAs, $oSaveAs $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs ) $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF ) ConsoleWrite( "$oSaveAs OK" & @CRLF ) $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 1000 ) EndFunc Note how similar the "Open File menu" code and the "Click Save As... menu" code is. Note also the long Sleep because it's a "Save As" dialog with a large number of elements. Add the code, close the previous Notepad (don't save), run the code and leave Notepad and Save As open. Save As window. Place mouse cursor over Save As title bar and press F2: ; --- Save As window --- ConsoleWrite( "--- Save As window ---" & @CRLF ) ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) EndFunc The rest of the automation tasks is about the Save As window. Therefore we want to use the Save As window as the starting point. Not Notepad. Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open. Set File name. Place mouse cursor over File name Edit control and press F1: ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld.txt" ) EndFunc $oSaveAsWin.FindFirst searches in the Save As window. $oEdit is now the File name Edit control in the Save As window. The file name is set to "HelloWorld.txt". Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open. Click Save button. Place mouse cursor over Save button and press F1: ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSave, $oSave $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave ) $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF ) ConsoleWrite( "$oSave OK" & @CRLF ) $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc Add the code, close the previous Save As and Notepad (don't save) and run the code. All code (Examples\Notepad\NotepadAll.au3): #include "..\..\Includes\CUIAutomation2.au3" Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Notepad window --- ConsoleWrite( "--- Notepad window ---" & @CRLF ) Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) ; --- Fill Edit element --- ConsoleWrite( "--- Fill Edit element ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) ; --- Click "Save As..." menu --- ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF ) ; Reuse $pCondition1 above ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAs, $oSaveAs $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs ) $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF ) ConsoleWrite( "$oSaveAs OK" & @CRLF ) $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 1000 ) ; --- Save As window --- ConsoleWrite( "--- Save As window ---" & @CRLF ) ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld.txt" ) ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSave, $oSave $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave ) $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF ) ConsoleWrite( "$oSave OK" & @CRLF ) $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc SciTE output: $oUIAutomation OK $oDesktop OK --- Notepad window --- $pCondition OK $oNotepad OK --- Fill Edit element --- $pCondition OK $oEdit OK $oValue OK --- Open File menu --- $pCondition1 OK $pCondition2 OK $pCondition OK $oFile OK $oInvoke OK --- Click "Save As..." menu --- $pCondition2 OK $pCondition OK $oSaveAs OK $oInvoke OK --- Save As window --- $pCondition OK $oSaveAsWin OK --- Set File name --- $pCondition OK $oEdit OK $oValue OK --- Click Save button --- $pCondition1 OK $pCondition2 OK $pCondition OK $oSave OK $oInvoke OK Maybe somebody can show how to implement such a Notepad automation with the functions in UIAWrappers.au3 and with classic code. Summary The two most important objects in UI Automation code are the $oUIAutomation and $oUIElement objects created from IUIAutomation and IUIAutomationElement interfaces. An UI Automation element object that represents a window or control is always created with the IUIAutomationElement interface through a pointer to the element object: $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) An UI Automation task is almost always done in two or three steps: 1. Create condition to find element 2. Find the UI Automation element 3. Extract info or perform action Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included). UI Automation actions (patterns) are performed through pattern interfaces, objects and methods. The code is nearly always implemented this way: Fill an Edit control: Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) $oEdit is an Edit control. The $oValue object represents the ValuePattern interface for the Edit control. $oValue.SetValue() sets the text in the Edit control. Click a menu item: Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) $oFile is a File menu. The $oInvoke object represents the InvokePattern interface for the File menu. $oInvoke.Invoke() clicks the File menu. Sleep( 100 ) gives the File menu time to open. Click a button control: $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() $oSave is a Save button. The $oInvoke object represents the InvokePattern interface for the Save button. $oInvoke.Invoke() clicks the Save button. When writing UI Automation code, it's easiest to write the code in small steps and test the code after each step. Error management based on checking objects with IsObj() and ConsoleWrite() is required and sufficient. It's very rare necessary to use an actual object (COM) error handler. Too small Sleep times after a menu, dialog or window is opened is a common error.1 point