Popular Post LarsJ Posted December 22, 2018 Popular Post Posted December 22, 2018 (edited) This example is about using Microsoft UI Automation APIs in AutoIt. Ie. it's about using interfaces, objects, methods, properties and constants as they are defined directly by Microsoft. In AutoIt, the definitions are contained in CUIAutomation2.au3 by junkew. The definitions are copied from corresponding C/C++ header files and customized the AutoIt language. This example is not about the functions in UIAWrappers.au3 by junkew. The purpose of the example is to show how to use Microsoft UI Automation APIs directly without any intermediate AutoIt functions. A major advantage of this approach is that the Microsoft documentation directly can be used as AutoIt documentation. Other major advantages are that there is no need for documenting and maintaining a set of intermediate AutoIt functions. Without intermediate AutoIt functions the code will be very fast. All Microsoft functions can be used this way eg. functions for event handling. New interfaces, objects, methods, properties and constants can easily be added simply by adding the definitions to CUIAutomation2.au3. Is this approach difficult? Not at all as the examples will show. The example is inspired by this comment of mLipok. Including documentation and pictures, even a simple Notepad automation can be somewhat comprehensive. Therefore a new example. First post will be a list of examples. The examples themselves are reviewed in the following posts. In all examples UIASpy is used to get element information. Topics Using UIASpy UIASpy thread Automating Notepad. Very detailed example. Automating Notepad with Sample code - step by step Automating Notepad with Sample code - all at once Automating Notepad - Windows XP Automating Notepad - Classic Click Save As... issue Automating Notepad context menu. Event handler example. Chrome - Clicking an extension. Compact summary example. Other spy tools Using Microsoft Inspect.exe simplespy.au3 by junkew How to topics If UI Automation or the UIASpy tool is new to you, then you should start reading theHow to topics Various topics Console and terminal windows Console application (3 posts) Terminal program (several posts) Examples Patterns (actions) Code snippets Real examples UIA updates UIA events Threads UI Automation UDFs contains all include files. UIASpy - UI Automation Spy Tool is a GUI tool that provides information about windows and controls and their interconnection and provides functionality to generate sample code. UIASpy is essential for creating UI Automation code. UI Automation Events is about implementing event handlers and includes GUIs to detect events. IUIAutomation MS framework automate chrome, FF, IE, .... created by junkew August 2013 is the first AutoIt thread on UIA code. Zip-file The zip contains source files for all examples. Note that UI Automation UDFs must be installed in the Includes folder. You need AutoIt 3.3.12 or later. Tested on Windows XP, Windows 7 and Windows 10. Comments and questions about using UIA code are welcome. Let me know if there are any issues. Questions about the functions in UIAWrappers.au3 should be asked in junkew's thread. UIAExamples.7z Edited February 9, 2020 by LarsJ Updates, new zip-file BatMan22, Trong, Gianni and 6 others 9 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
Popular Post LarsJ Posted December 22, 2018 Author Popular Post Posted December 22, 2018 (edited) Automating Notepad This example is a response to a request by mLipok for a simple Notepad automation. Task: Create a simple script to fill up "Edit1" with HelloWorld, click "Save As...", enter filename and finally save it to disc. This involves automating Notepad or more generally an Edit control, it involves automating a menu and a Save As dialog. Note that "Edit1" (from AutoIt Window Info tool) is not a valid control identification in UI Automation code. Do not open Notepad in advance. This will be done in the code. Start the code by opening Notepad and create the UI Automation object through IUIAutomation interface. I almost always create the Desktop element. Note how you first get a pointer to the element (GetRootElement method of $oUIAutomation object), and then creates the object that represents the element through IUIAutomationElement interface. UI Automation elements are nearly always created this way through a pointer. #include "..\..\Includes\CUIAutomation2.au3" Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc Open Examples\Notepad\Notepad.au3 (empty) in SciTE. Add the code, run the code with F5 and leave Notepad open. Check that SciTE output is OK. Next step is to get the Notepad window as an UI Automation element. Open UIASpy (included in zip-file, keep UIASpy open throughout the entire example), place mouse cursor over Notepad title bar and press F2: (You can delete other windows in UIASpy by right-clicking treeview top windows (level one items).) $UIA_ClassNamePropertyId = "Notepad" is used to create a property condition to identify Notepad. Note that you can copy this line in UIASpy listview: Select the line in the listview, click "Right pane" item in main menu, click "Copy all or selected items to clipboard", paste the line into your code editor. Add this code to the existing code: ; --- Notepad window --- ConsoleWrite( "--- Notepad window ---" & @CRLF ) Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) EndFunc Note again how we first get a pointer to Notepad (FindFirst method of $oDesktop object, $oDesktop because the Desktop is the parent element of Notepad) and then creates the object. Also note that $oDesktop and $oNotepad are created in exactly the same way. Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. Now, the actual automation starts: Fill up "Edit1" with HelloWorld. An automation task is almost always done in two or three steps: 1. Create condition to find element 2. Find the UI Automation element 3. Extract info or perform action Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included). To get the Notepad window above we only used two steps. The first automation task is to perform an action to fill out the Edit control with "HelloWorld". Place mouse cursor in the Edit control and press F1: (If there are two or more Notepad windows in UIASpy, you can delete the old windows (red) by right-clicking the treeview.) This time we can use $UIA_AutomationIdPropertyId = "15" to create a property condition to identify the Edit control. Add this code: ; --- Fill Edit element --- ConsoleWrite( "--- Fill Edit element ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) EndFunc $oNotepad.FindFirst means that the search for the Edit control starts in the Notepad window. $oEdit reprecents the Edit control. Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. The Edit control must be filled with "HelloWorld". This is an action that has to be performed on the Edit control. The following actions are available: (Scroll down UIASpy listview to see Control Patterns (actions).) The interesting action or pattern here is the ValuePattern. The action is created and performed this way: Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) EndFunc $oEdit.GetCurrentPattern means that a pattern is created that performs actions on the Edit control. GetCurrentPattern method returns a pointer where from the ValuePattern object ($oValue) can be created. $UIA_ValuePatternId and the ValuePattern interface is defined in CUIAutomation2.au3. The $oValue object represents the ValuePattern interface for the Edit control. $oValue.SetValue( "HelloWorld" ) sets the text in the Edit control. An action or pattern is almost always created and performed this way. Add the code, close the previous Notepad, run the code and leave Notepad open. Open File menu. Place mouse cursor over the File menu and press F1: (Scroll down UIASpy listview to see Control Patterns (actions) in bottom of image.) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) EndFunc The Invoke action or pattern is used to click and open the File menu. Note the similarity between the code for InvokePattern and ValuePattern above. When a menu, dialog box or child window is opened it need some time to open. You must add a Sleep statement. Add the code, change "File" to the corresponding word in your own language (copy the word from the listview), close the previous Notepad (don't save), run the code and leave Notepad open. Click "Save As..." menu. If the File menu that was opened above is closed, then open it manually with a mouse click. Place mouse cursor over the "Save As..." menu and press F1: ; --- Click "Save As..." menu --- ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF ) ; Reuse $pCondition1 above ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAs, $oSaveAs $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs ) $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF ) ConsoleWrite( "$oSaveAs OK" & @CRLF ) $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 1000 ) EndFunc Note how similar the "Open File menu" code and the "Click Save As... menu" code is. Note also the long Sleep because it's a "Save As" dialog with a large number of elements. Add the code, close the previous Notepad (don't save), run the code and leave Notepad and Save As open. Save As window. Place mouse cursor over Save As title bar and press F2: ; --- Save As window --- ConsoleWrite( "--- Save As window ---" & @CRLF ) ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) EndFunc The rest of the automation tasks is about the Save As window. Therefore we want to use the Save As window as the starting point. Not Notepad. Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open. Set File name. Place mouse cursor over File name Edit control and press F1: ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld.txt" ) EndFunc $oSaveAsWin.FindFirst searches in the Save As window. $oEdit is now the File name Edit control in the Save As window. The file name is set to "HelloWorld.txt". Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open. Click Save button. Place mouse cursor over Save button and press F1: ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSave, $oSave $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave ) $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF ) ConsoleWrite( "$oSave OK" & @CRLF ) $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc Add the code, close the previous Save As and Notepad (don't save) and run the code. All code (Examples\Notepad\NotepadAll.au3): expandcollapse popup#include "..\..\Includes\CUIAutomation2.au3" Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Notepad window --- ConsoleWrite( "--- Notepad window ---" & @CRLF ) Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) ; --- Fill Edit element --- ConsoleWrite( "--- Fill Edit element ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) ; --- Click "Save As..." menu --- ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF ) ; Reuse $pCondition1 above ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAs, $oSaveAs $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs ) $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF ) ConsoleWrite( "$oSaveAs OK" & @CRLF ) $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 1000 ) ; --- Save As window --- ConsoleWrite( "--- Save As window ---" & @CRLF ) ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld.txt" ) ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSave, $oSave $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave ) $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF ) ConsoleWrite( "$oSave OK" & @CRLF ) $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc SciTE output: $oUIAutomation OK $oDesktop OK --- Notepad window --- $pCondition OK $oNotepad OK --- Fill Edit element --- $pCondition OK $oEdit OK $oValue OK --- Open File menu --- $pCondition1 OK $pCondition2 OK $pCondition OK $oFile OK $oInvoke OK --- Click "Save As..." menu --- $pCondition2 OK $pCondition OK $oSaveAs OK $oInvoke OK --- Save As window --- $pCondition OK $oSaveAsWin OK --- Set File name --- $pCondition OK $oEdit OK $oValue OK --- Click Save button --- $pCondition1 OK $pCondition2 OK $pCondition OK $oSave OK $oInvoke OK Maybe somebody can show how to implement such a Notepad automation with the functions in UIAWrappers.au3 and with classic code. Summary The two most important objects in UI Automation code are the $oUIAutomation and $oUIElement objects created from IUIAutomation and IUIAutomationElement interfaces. An UI Automation element object that represents a window or control is always created with the IUIAutomationElement interface through a pointer to the element object: $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) An UI Automation task is almost always done in two or three steps: 1. Create condition to find element 2. Find the UI Automation element 3. Extract info or perform action Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included). UI Automation actions (patterns) are performed through pattern interfaces, objects and methods. The code is nearly always implemented this way: Fill an Edit control: Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) $oEdit is an Edit control. The $oValue object represents the ValuePattern interface for the Edit control. $oValue.SetValue() sets the text in the Edit control. Click a menu item: Local $pInvoke, $oInvoke $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 100 ) $oFile is a File menu. The $oInvoke object represents the InvokePattern interface for the File menu. $oInvoke.Invoke() clicks the File menu. Sleep( 100 ) gives the File menu time to open. Click a button control: $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() $oSave is a Save button. The $oInvoke object represents the InvokePattern interface for the Save button. $oInvoke.Invoke() clicks the Save button. When writing UI Automation code, it's easiest to write the code in small steps and test the code after each step. Error management based on checking objects with IsObj() and ConsoleWrite() is required and sufficient. It's very rare necessary to use an actual object (COM) error handler. Too small Sleep times after a menu, dialog or window is opened is a common error. Edited December 22, 2018 by LarsJ ss26, Decibel, ioa747 and 10 others 9 4 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
InnI Posted December 22, 2018 Posted December 22, 2018 @LarsJ Why don't you create a condition object from a pointer and use the pointer directly? Local $pCondition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) ; use pointer What's the difference between your code and this? Local $pCondition, $oCondition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer $oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition ) ; create object Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad ) ; use object
mLipok Posted December 22, 2018 Posted December 22, 2018 (edited) This is what I was looking for. BIG THANKS @LarsJ Edited December 22, 2018 by mLipok Signature beginning:* Please remember: "AutoIt"..... * Wondering who uses AutoIt and what it can be used for ? * Forum Rules ** ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Code * for other useful stuff click the following button: Spoiler Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API * ErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) * Show_Macro_Values.au3 * My contribution to others projects or UDF based on others projects: * _sql.au3 UDF * POP3.au3 UDF * RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF * SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane * Useful links: * Forum Rules * Forum etiquette * Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * Wiki: * Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX IE Related: * How to use IE.au3 UDF with AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler * IE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related: * How to get reference to PDF object embeded in IE * IE on Windows 11 * I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions * EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *I also encourage you to check awesome @trancexx code: * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuff * OnHungApp handler * Avoid "AutoIt Error" message box in unknown errors * HTML editor * winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/ "Homo sum; humani nil a me alienum puto" - Publius Terentius Afer"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming" , be and \\//_. Anticipating Errors : "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty." Signature last update: 2023-04-24
junkew Posted December 22, 2018 Posted December 22, 2018 Nice detailed example and yes uiautomation of ms is in general fast when you can do exact matching. Similar examples are given in examples.zip of uiautomation thread. Main reason for the uiawrappers is to have more flexibility on Index, indexrelative Small wrappers around pointer and objcreateinterface logic Named properties and easy combining them At some cost of speed Setvalue, settext interface on many objects not working Invoke sometimes is a click and sometimes a setfocus. Match on regular expressions FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
LarsJ Posted December 23, 2018 Author Posted December 23, 2018 (edited) InnI, Because these 2 ConsoleWrites prints the same value ConsoleWrite( "Ptr( $pCondition ) = " & Ptr( $pCondition ) & @CRLF ) ConsoleWrite( "Ptr( $oCondition() ) = " & Ptr( $oCondition() ) & @CRLF ) then I don't think there is much difference between these lines: $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad ) $oDesktop.FindFirst( $TreeScope_Descendants, $oCondition(), $pNotepad ) But you are using an additional line to create the $oCondition object. Since you are not calling any methods of the object, it seems to be an unnecessary additional line. junkew, There are none of these points that cannot be implemented even if one uses a direct approach as demonstrated here. Using the Microsoft code in a direct way is simply not a limitation. You can make an example or code snippet covering each situation. I have already made a number of examples based on this technique, most of them in your own thread. There is nothing that can't be done. Edited December 23, 2018 by LarsJ Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
Earthshine Posted December 23, 2018 Posted December 23, 2018 You guys rule. This makes my Christmas. Thanks LarsJ and junkew both for making my automated life better! My resources are limited. You must ask the right questions
InnI Posted December 23, 2018 Posted December 23, 2018 1 hour ago, LarsJ said: it seems to be an unnecessary additional line Could you please explain, why there is a constant increase in memory usage in this code? #include "CUIAutomation2.au3" HotKeySet("{esc}", "Quit") Func Quit() Exit EndFunc Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) Global $pCondition, $pDesktop, $oDesktop, $pProgman, $oProgman $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) While 1 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition ) $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pProgman ) $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj($oProgman) Then Exit WEnd But in this code, memory usage increases slightly and remains constant #include "CUIAutomation2.au3" HotKeySet("{esc}", "Quit") Func Quit() Exit EndFunc Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) Global $pCondition, $oCondition, $pDesktop, $oDesktop, $pProgman, $oProgman $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) While 1 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition ) $oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition ) $oDesktop.FindFirst( $TreeScope_Children, $oCondition, $pProgman ) $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj($oProgman) Then Exit WEnd It looks like a memory leak occurs when to use the pointer directly.
argumentum Posted December 23, 2018 Posted December 23, 2018 When I investigate SciTE, ( with any UI tool ) it crashes when I try to "F5" to run a script. So I guess SciTE is not UIA friendly. Question: is there a way to ask an executable if is UIA friendly ? Follow the link to my code contribution ( and other things too ). FAQ - Please Read Before Posting.
junkew Posted December 23, 2018 Posted December 23, 2018 What do you mean with UIA friendly? Automating SCITE works fine for me (copy, pasted with simplespy merged and some small fixes on patterns for recognition) ;~ *** Standard code Flexible*** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=SciTE;controltype:=UIA_WindowControlTypeId;class:=SciTEWindow", $treescope_children) ;~ _UIA_Action($oP1,"setfocus") _UIA_action($oP1,"highlight") Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Toepassing;controltype:=UIA_MenuBarControlTypeId;class:=", $treescope_children) ;~ First find the object in the parent before you can do something ;~ $oUIElement=_UIA_getObjectByFindAll("Tools.mainwindow", "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"highlight") _UIA_action($oUIElement,"click") Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Tools;controltype:=UIA_MenuControlTypeId;class:=#32768", $treescope_children) _UIA_Action($oP0,"setfocus") _UIA_Action($oP0,"highlight") ;~ First find the object in the parent before you can do something ;~ $oUIElement=_UIA_getObjectByFindAll("Go F5.mainwindow", "title:=Go F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Go F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"highlight") ;~ _UIA_action($oUIElement,"click") FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
argumentum Posted December 24, 2018 Posted December 24, 2018 (edited) On 12/23/2018 at 3:33 PM, junkew said: What do you mean with UIA friendly? Automating SCITE works fine for me hmmm, I'm trying to replicate the crashes I experienced before, ...but it wont crash. Tho my PC rebooted due to windows update. Maybe all it needed was a reboot. Sowy, disregard my question and thanks for the answer =) PS: ...I'll look into it. PS2: in SciTE v3.6.6 there is no crash but in SciTE v4.1.0 it crashes with the above Pic. Be aware that I am not talking about running code. The crash is after I use, any tool really, to highlight the ID/Text/etc., and press F5 to run any code from the editor, is that the editor crashes. PS3: use "SampleControls.au3" from your hard drive to replicate. Edited December 25, 2018 by argumentum deep, ( or not ), investigation result Follow the link to my code contribution ( and other things too ). FAQ - Please Read Before Posting.
LarsJ Posted December 24, 2018 Author Posted December 24, 2018 InnI, I can't explain that. But in relation to UI Automation code, it is not really a problem because we're dealing with a very limited number of objects. Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
junkew Posted January 1, 2019 Posted January 1, 2019 (edited) Based on something you wrote in 2014 in UIAutomation thread I am curious on if this still is a valid statement? why? Any example where this has to be done on a propertycondition? Quote If you are running a 32 bit script, you must change the CreatePropertyCondition description in this way: "CreatePropertyCondition hresult(int;uint64;uint64;ptr*);" If you are running a 64 bit script, you must change the CreatePropertyCondition description in this way: "CreatePropertyCondition hresult(int;ptr;ptr*);" You need this description for an advanced data type like a structure (created with DllStructCreate). For simple data types like numbers and strings, you still use the old description with a variant. The reason for two new defitions are caused by differences in parameter transfer on 32 and 64 bit systems. I am working on (MIDL to AU3 converter) instead of CUIAutomation2.au3 we will have UIAutomationClient.au3 based on the IDL UIAutomationClient.IDL and then I will generate ;~ HRESULT CreatePropertyCondition ([in] PROPERTYID propertyId, [in] VARIANT value, [out, retval] IUIAutomationCondition ** newCondition ); "CreatePropertyCondition hresult(struct;variant;ptr*);" Edited January 1, 2019 by junkew LarsJ 1 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
LarsJ Posted January 2, 2019 Author Posted January 2, 2019 This is still valid. It must be taken into account in situations where a method takes an array as input parameter and the array is not an array of variants. If you search for VT_ARRAY in this list you'll get 13 matches. When you use any of these PropertyIds as input parameters you have to handle the arrays properly. The problem arises because the internal AutoIt COM conversions that converts native AutoIt variables and arrays to COM compatible data types always convert a native AutoIt array into a safearray of variants. But in these 13 situations the arrays are safearrays of integers, doubles and IUnknown pointers and thus not safearrays of variants. Here we have to handle the COM conversions ourselves. When a safearray is passed as a parameter to a COM method it's usually passed as a variant ($tagVariant structure) where the variant data field contains a pointer to the safearray. In 32 bit code parameters are passed through the stack. In 32 bit code a variant takes up 16 bytes. The purpose of "uint64;uint64;" is to pass the 16 bytes variant structure onto the stack wherefrom it's read by the internal method code. In 64 bit code the first 4 parameters are passed through CPU registers. If one of the first 4 parameters is a structure and the structure isn't larger than 8 bytes (64 bit) it's passed directly through the CPU register. If it's larger than 8 bytes a pointer to the structure is passed through the CPU register. In 64 bit code a variant takes up 24 bytes. The purpose of "ptr;" is to pass a pointer to the variant structure through a CPU register wherefrom the pointer and then the variant structure is read by the internal method code. If you read the posts around the quoted post, you'll see a working example. junkew, Let's make a little deal. I'll code a function to handle arrays as input parameters and post it here. Then you can copy the function directly into your new UDF. In return, I'll use your UDF in my code. Earthshine and junkew 2 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
junkew Posted January 2, 2019 Posted January 2, 2019 thx for the explanation. Deal is great The new UIA UDF will be based on the UIA IDL files of Win10 SDK, (So I think 2-3 files) generated by the converterhttps://www.autoitscript.com/forum/topic/197177-idl-to-au3-generator/ Little in doubt if the generator is just good enough and some manual changes to be done afterward on generated au3 files (alternative is to build IDL lexer/parser but more work or use OAIdl.Idl to make an .au3 file and use the typelib info to generate then the au3 files) UIAWrappers will be then based on definitions of step 1 improved on finding on single propertyconditions (input you gave in this thread and UIA thread) where no regex finding is needed Extended simplespy with some addition coding examples when spied on object fixed/cleaned examples mLipok 1 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
LarsJ Posted January 25, 2019 Author Posted January 25, 2019 (edited) Automating Notepad with Sample code - step by step This is a repetition of the Notepad example above. But this time, the possibilities for creating Sample code in UIASpy will be exploited. Only the first part of the example will be reviewed once more in detail. But the pictures will not be repeated. So please open the previous example in a new tab in the browser. Open "Examples\Notepad\Windows 7, 10\2) Sample code - step by step\Notepad.au3" (empty) in SciTE. The file is included in the zip-file in bottom of first post. Open UIASpy (also included in zip-file) and keep UIASpy open throughout the entire example. Open Left pane main menu and click Delete top windows to delete all treeview windows except Desktop. First step is to add initial code to create the UI Automation object and the Desktop element. Open Sample code main menu in UIASpy and click Initial code. Open the menu once more and click Corrections to get an idea of the corrections that have to be made to this generic code. Paste the code from clipboard into SciTE (Shift+Insert). The code should look like this: #AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc ; Code corrections: ; CUIAutomation2.au3 path ; Code to open target appl ; Create undeclared variables ; Delete double declared variables ; Correct $pWindow, $pParent and $pControl names ; Correct $oWindow, $oParent and $oControl names Correct the path to CUIAutomation2.au3. Add code to open Notepad. Delete correction comments. Now the code should look like this: #AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "..\..\..\..\Includes\CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc Run the code with F5 and leave Notepad open. Check that SciTE output is OK. Next step is to get the Notepad window as an UI Automation element. Place mouse cursor over Notepad title bar and press F2. Select the row in the Notepad detail information listview page in UIASpy that matches the row with the red arrow in the first picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code. Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code. Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this: ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pWindow, $oWindow $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF ) ; Code corrections: ; CUIAutomation2.au3 path ; Code to open target appl ; Create undeclared variables ; Delete double declared variables ; Correct $pWindow, $pParent and $pControl names ; Correct $oWindow, $oParent and $oControl names Replace "window/control" with "Notepad window". Replace "Window" with "Notepad". Delete correction comments. Move code into function: ; --- Find Notepad window --- ConsoleWrite( "--- Find Notepad window ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. Third step is to fill out the Edit control with "HelloWorld". An UI Automation task is almost always done in two or three steps: 1. Create condition to find element 2. Find the UI Automation element 3. Extract info or perform action Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included). To get the Notepad window above we only used two steps. Place mouse cursor in the Edit control in Notepad and press F1. Select the row in the Edit detail information listview page in UIASpy that matches the row with the red arrow in the second picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code. Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code. Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this: ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pEdit, $oEdit $oParent.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) ; Code corrections: ; CUIAutomation2.au3 path ; Code to open target appl ; Create undeclared variables ; Delete double declared variables ; Correct $pWindow, $pParent and $pControl names ; Correct $oWindow, $oParent and $oControl names Replace "Find window/control" with "Fill Edit control". Delete the line "Local $pCondition0". Replace "$oParent" with "$oNotepad". Delete correction comments. Move code into function: ; --- Fill Edit control --- ConsoleWrite( "--- Fill Edit control ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK. Place mouse cursor in the Edit control in Notepad and press F1. If there is a red (invalid) Notepad window in the treeview then delete it (Right-click the red element and click Delete element and all childs). Scroll down to the Control Patterns (element actions) section in the Edit detail information listview page in UIASpy. Select the row "$UIA_IsValuePatternAvailablePropertyId" that matches the row with the red arrow in the third picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code. Click the Edit item in the treeview to return to the Edit detail information listview page. Scroll down to the bottom of the listview page. The Value Pattern Methods subsection should be visible. Select the row with the SetValue(bstr) method in second column: Right-click and click Create sample code (don't clear the code this time). Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this: ; --- Value Pattern (action) Object --- ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF ) Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) ; --- Value Pattern (action) Methods --- ConsoleWrite( "--- Value Pattern (action) Methods ---" & @CRLF ) $oValuePattern.SetValue(bstr) Delete the two comments, two ConsoleWrites and empty lines. Replace "bstr" with "HelloWorld". Move code into function: Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue( "HelloWorld" ) Close the previous Notepad, run the code, check that SciTE output is OK, check "HelloWorld" in Notepad Edit control. All code after the third step: expandcollapse popup#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "..\..\..\..\Includes\CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Find Notepad window --- ConsoleWrite( "--- Find Notepad window ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) ; --- Fill Edit control --- ConsoleWrite( "--- Fill Edit control ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue( "HelloWorld" ) EndFunc SciTE output: $oUIAutomation OK $oDesktop OK --- Find Notepad window --- $pCondition0 OK $oNotepad OK --- Fill Edit control --- $pCondition0 OK $oEdit OK $oValuePattern OK Edited January 25, 2019 by LarsJ yutijang and Decibel 1 1 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
LarsJ Posted January 25, 2019 Author Posted January 25, 2019 (edited) Automating Notepad with Sample code - all at once In an automation task like the Notepad example above, it's much faster to create all sample code at once than it is to create the sample code in a step-by-step procedure. For pattern methods, it's also faster to create sample code by selecting the method through the Pattern... item in the Sample code menu in UIASpy than it is to create pattern method sample code through the detail information listview page. But to use the pattern listview page you need to know which pattern to use. The advantage of using the detail information listview page is that it only shows the patterns and pattern methods that are applicable to the specific control. These are all the steps in the Notepad example: Add initial code Open Notepad Sleep( 1000 ) Get Notepad window Fill Edit control Identify Edit control Execute SetValue pattern method Open File menu Identify File menu Execute Invoke pattern method Sleep( 100 ) Click Save As... menu item Identify Save As... menu item Execute Invoke pattern method Sleep( 1000 ) Get Save As window Fill Edit control (File name) Identify Edit control Execute SetValue pattern method Click Save button Identify Save button Execute Invoke pattern method Corrections And this is sample code for all the steps in the entire Notepad example. Examples\Notepad\Windows 7, 10\3) Sample code - all at once\1) Before corrections.au3: expandcollapse popup#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pWindow, $oWindow $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF ) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pEdit, $oEdit $oParent.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) ; --- Value Pattern (action) Object --- ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF ) Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue(bstr) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition2, $pCondition3, $pAndCondition3 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 ) $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 ) If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition3 OK" & @CRLF ) Local $pMenuItem, $oMenuItem $oParent.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() Sleep( 1000 ) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition4, $pCondition5, $pAndCondition5 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 ) $oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 ) If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition5 OK" & @CRLF ) Local $pMenuItem, $oMenuItem $oParent.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() Sleep( 1000 ) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition6 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 ) If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF ) ConsoleWrite( "$pCondition6 OK" & @CRLF ) Local $pWindow, $oWindow $oParent.FindFirst( $TreeScope_Descendants, $pCondition6, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF ) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition7 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 ) If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF ) ConsoleWrite( "$pCondition7 OK" & @CRLF ) Local $pEdit, $oEdit $oParent.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) ; --- Value Pattern (action) Object --- ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF ) Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue(bstr) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition8, $pCondition9, $pAndCondition9 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 ) $oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 ) If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition9 OK" & @CRLF ) Local $pButton, $oButton $oParent.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton ) $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF ) ConsoleWrite( "$oButton OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() ; Code corrections: ; CUIAutomation2.au3 path ; Code to open target appl ; Create undeclared variables ; Delete double declared variables ; Correct $pWindow, $pParent and $pControl names ; Correct $oWindow, $oParent and $oControl names The code isn't immediately executable. It's necessary to make some corrections. Most corrections are listed at the bottom of the code as comments. Note that all condition variables are created properly. They need no corrections. This is the code after corrections. Examples\Notepad\Windows 7, 10\3) Sample code - all at once\2) After corrections.au3: expandcollapse popup#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "..\..\..\..\Includes\CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Get Notepad window --- ConsoleWrite( "--- Get Notepad window ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) ; --- Fill Edit control --- ConsoleWrite( "--- Fill Edit control ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Local $pValuePattern, $oValuePattern $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue( "HelloWorld" ) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition2, $pCondition3, $pAndCondition3 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 ) ; <<<<<<<<<<<<<<<<<<<<<<<< $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 ) If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition3 OK" & @CRLF ) Local $pMenuItem, $oMenuItem $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() Sleep( 100 ) ; --- Click Save As... menu item --- ConsoleWrite( "--- Click Save As... menu item ---" & @CRLF ) Local $pCondition4, $pCondition5, $pAndCondition5 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 ) $oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 ) If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition5 OK" & @CRLF ) $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() Sleep( 1000 ) ; --- Get Save As window --- ConsoleWrite( "--- Get Save As window ---" & @CRLF ) Local $pCondition6 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 ) If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF ) ConsoleWrite( "$pCondition6 OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition6, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) ; --- Fill Edit control (File name) --- ConsoleWrite( "--- Fill Edit control (File name) ---" & @CRLF ) Local $pCondition7 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 ) If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF ) ConsoleWrite( "$pCondition7 OK" & @CRLF ) $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern ) $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF ) ConsoleWrite( "$oValuePattern OK" & @CRLF ) $oValuePattern.SetValue( "HelloWorld.txt" ) ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) Local $pCondition8, $pCondition9, $pAndCondition9 $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 ) $oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 ) If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition9 OK" & @CRLF ) Local $pButton, $oButton $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton ) $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF ) ConsoleWrite( "$oButton OK" & @CRLF ) $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() EndFunc When executing the code, it should perform all Notepad automation tasks and generate this output in SciTE. SciTE output: $oUIAutomation OK $oDesktop OK --- Get Notepad window --- $pCondition0 OK $oNotepad OK --- Fill Edit control --- $pCondition1 OK $oEdit OK $oValuePattern OK --- Open File menu --- $pAndCondition3 OK $oMenuItem OK $oInvokePattern OK --- Click Save As... menu item --- $pAndCondition5 OK $oMenuItem OK $oInvokePattern OK --- Get Save As window --- $pCondition6 OK $oSaveAsWin OK --- Fill Edit control (File name) --- $pCondition7 OK $oEdit OK $oValuePattern OK --- Click Save button --- $pAndCondition9 OK $oButton OK $oInvokePattern OK Edited January 26, 2019 by LarsJ yutijang and junkew 2 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
LarsJ Posted January 26, 2019 Author Posted January 26, 2019 (edited) Automating Notepad - Windows XP This is a repetition of the Notepad example above. But this time under Windows XP. I'll not review the whole example once more but only focus on the challenges of Windows XP. These challenges are about not all pattern methods working under Windows XP. There are two cases where the pattern methods does not work. The first case is to open the File menu by clicking the menu item. Here, the Invoke method does not work. The only thing you can do is use the bounding rectangle to position the mouse over the File menu and perform a mouse click: ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) ; Rectangle Local $aRect ; l, t, w, h $oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect ) If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF ) ConsoleWrite( "$aRect OK" & @CRLF ) ; Click File menu MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 ) Sleep( 100 ) The second case is to fill out the File name (an Edit control) in the "Save As ..." dialog. The SetValue pattern method does not work in this Edit control. The only thing that works is the Send command. Note that the File name control gets focus when the "Save As ..." dialog opens and that the text in Edit control is selected: ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ConsoleWrite( "Send()" & @CRLF ) Send( "HelloWorld.txt", 1 ) All Windows XP code. Examples\Notepad\Windows XP\1) Create code\NotepadAll.au3: expandcollapse popup#include "..\..\..\..\Includes\CUIAutomation2.au3" Example() Func Example() ; Open Notepad Run( "Notepad" ) Sleep( 1000 ) ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Notepad window --- ConsoleWrite( "--- Notepad window ---" & @CRLF ) Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pNotepad, $oNotepad $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF ) ConsoleWrite( "$oNotepad OK" & @CRLF ) ; --- Fill Edit element --- ConsoleWrite( "--- Fill Edit element ---" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pEdit, $oEdit $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit ) $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF ) ConsoleWrite( "$oEdit OK" & @CRLF ) Local $pValue, $oValue $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue ) $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern ) If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF ) ConsoleWrite( "$oValue OK" & @CRLF ) $oValue.SetValue( "HelloWorld" ) ; --- Open File menu --- ConsoleWrite( "--- Open File menu ---" & @CRLF ) Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<< If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pFile, $oFile $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile ) $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF ) ConsoleWrite( "$oFile OK" & @CRLF ) ; Rectangle Local $aRect ; l, t, w, h $oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect ) If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF ) ConsoleWrite( "$aRect OK" & @CRLF ) ; Click File menu MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 ) Sleep( 100 ) ; --- Click "Save As..." menu --- ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF ) ; Reuse $pCondition1 above ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAs, $oSaveAs $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs ) $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF ) ConsoleWrite( "$oSaveAs OK" & @CRLF ) Local $pInvoke, $oInvoke $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() Sleep( 1000 ) ; --- Save As window --- ConsoleWrite( "--- Save As window ---" & @CRLF ) ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSaveAsWin, $oSaveAsWin, $hSaveAsWin $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin ) $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) $oSaveAsWin.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hSaveAsWin ) If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF ) ConsoleWrite( "$oSaveAsWin OK" & @CRLF ) ; --- Set File name --- ConsoleWrite( "--- Set File name ---" & @CRLF ) ConsoleWrite( "Send()" & @CRLF ) Send( "HelloWorld.txt", 1 ) ; --- Click Save button --- ConsoleWrite( "--- Click Save button ---" & @CRLF ) $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) Local $pSave, $oSave $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave ) $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF ) ConsoleWrite( "$oSave OK" & @CRLF ) $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc SciTE output: $oUIAutomation OK $oDesktop OK --- Notepad window --- $pCondition OK $oNotepad OK --- Fill Edit element --- $pCondition OK $oEdit OK $oValue OK --- Open File menu --- $pCondition1 OK $pCondition2 OK $pCondition OK $oFile OK $aRect OK --- Click "Save As..." menu --- $pCondition2 OK $pCondition OK $oSaveAs OK $oInvoke OK --- Save As window --- $pCondition OK $oSaveAsWin OK --- Set File name --- Send() --- Click Save button --- $pCondition1 OK $pCondition2 OK $pCondition OK $oSave OK $oInvoke OK Edited January 26, 2019 by LarsJ Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
LarsJ Posted January 27, 2019 Author Posted January 27, 2019 (edited) Chrome - Clicking an extension A day ago, there was a question of clicking a Chrome extension. This is a demonstration of how it can be done with UIASpy. Here the "Blank New Tab Page" button is clicked. Chrome is already open and displaying the AutoIt start page. Because this is about the outer elements of Chrome it's important that accessibility is not enabled. Open UIASpy, turn off Only visible elements in the Left pane menu, place the mouse cursor over the "Blank New Tab Page" button and press F1: The butten is located below a pane control. That's why Only visible elements has to be turned off. This is also the reason why the button cannot be identified with the AutoIt Window Info tool. And junkew's simplespy.au3 will identify the pane control instead of the button. So we are not at all interested in the pane control. We are interested in the Chrome top window: Open Sample code main menu, click Initial code, click the selected treeview item to return to the detail info page, right click the selected row and click Create sample code. Then we are interested in the "Blank New Tab Page" button: Repeat this for each group of selected row(s): Right click a selected row and click Create sample code. Click the "Blank New Tab Page" button in Chrome to open a menu (see the picture above). Place the mouse cursor over the "Blank New Tab Page" menu item and press F1. The menu is closed when UIASpy gets focus. That's why the treeview items are red: Repeat this for each group of selected row(s): Right click a selected row and click Create sample code. Because I've tested the code, I know that the Invoke method doesn't work for the "Blank New Tab Page" menu item. We have to use a mouse click: Right click the selected row and click Create sample code. Open Sample code main menu, click Corrections. Paste the code into an editor. All sample code (Examples\Chrome\Click extension button\1) Before corrections.au3) should look like this: expandcollapse popup#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) EndFunc ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pPane, $oPane $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pPane ) $oPane = ObjCreateInterface( $pPane, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oPane ) Then Return ConsoleWrite( "$oPane ERR" & @CRLF ) ConsoleWrite( "$oPane OK" & @CRLF ) ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition1, $pCondition2, $pAndCondition2 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 ) $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 ) If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition2 OK" & @CRLF ) Local $pButton, $oButton $oParent.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton ) $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF ) ConsoleWrite( "$oButton OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) ; --- Invoke Pattern (action) Methods --- ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF ) $oInvokePattern.Invoke() ; --- Find window/control --- ConsoleWrite( "--- Find window/control ---" & @CRLF ) Local $pCondition3, $pCondition4, $pAndCondition4 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 ) $oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 ) If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition4 OK" & @CRLF ) Local $pMenuItem, $oMenuItem $oParent.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() ; --- Element Properties (information) --- ConsoleWrite( "--- Element Properties (information) ---" & @CRLF ) Local $vValue $oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $vValue ) ; Code corrections: ; CUIAutomation2.au3 path ; Code to open target appl ; Create undeclared variables ; Delete double declared variables ; Correct $pWindow, $pParent and $pControl names ; Correct $oWindow, $oParent and $oControl names Code after corrections (Examples\Chrome\Click extension button\2) After corrections.au3): expandcollapse popup#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 ;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code ;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code #include "..\..\..\Includes\CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Create UI Automation object Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get Desktop element Local $pDesktop, $oDesktop $oUIAutomation.GetRootElement( $pDesktop ) $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF ) ConsoleWrite( "$oDesktop OK" & @CRLF ) ; --- Find Chrome window --- ConsoleWrite( "--- Find Chrome window ---" & @CRLF ) Local $pCondition0 $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 ) If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF ) ConsoleWrite( "$pCondition0 OK" & @CRLF ) Local $pChrome, $oChrome $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pChrome ) $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF ) ConsoleWrite( "$oChrome OK" & @CRLF ) ; --- Find "Blank New Tab Page" button --- ConsoleWrite( "--- Find ""Blank New Tab Page"" button ---" & @CRLF ) Local $pCondition1, $pCondition2, $pAndCondition2 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 ) $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 ) If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition2 OK" & @CRLF ) Local $pButton, $oButton $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton ) $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF ) ConsoleWrite( "$oButton OK" & @CRLF ) ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) Local $pInvokePattern, $oInvokePattern $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) ; --- Invoke Pattern (action) Methods --- ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF ) $oInvokePattern.Invoke() Sleep( 100 ) ; --- Find "Blank New Tab Page" menu item --- ConsoleWrite( "--- Find ""Blank New Tab Page"" menu item ---" & @CRLF ) Local $pCondition3, $pCondition4, $pAndCondition4 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 ) $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 ) $oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 ) If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF ) ConsoleWrite( "$pAndCondition4 OK" & @CRLF ) Local $pMenuItem, $oMenuItem $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem ) $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF ) ConsoleWrite( "$oMenuItem OK" & @CRLF ) #cs ; --- Invoke Pattern (action) Object --- ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF ) $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern ) $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF ) ConsoleWrite( "$oInvokePattern OK" & @CRLF ) $oInvokePattern.Invoke() #ce ; Rectangle Local $aRect ; l, t, w, h $oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect ) If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF ) ConsoleWrite( "$aRect OK" & @CRLF ) ; Click menu item MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 ) EndFunc SciTE output: $oUIAutomation OK $oDesktop OK --- Find Chrome window --- $pCondition0 OK $oChrome OK --- Find "Blank New Tab Page" button --- $pAndCondition2 OK $oButton OK --- Invoke Pattern (action) Object --- $oInvokePattern OK --- Invoke Pattern (action) Methods --- --- Find "Blank New Tab Page" menu item --- $pAndCondition4 OK $oMenuItem OK $aRect OK New zip-file at bottom of first post. Edited January 31, 2019 by LarsJ A few text updates Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions
junkew Posted January 28, 2019 Posted January 28, 2019 Nice examples. Simplespy just uses elementfrompoint and you could argue pane as child from button is not a logical gui tree. Similar behavior you can see when automating ms access forms. An improved spy i tried scan hierarchy elementfrompoint up to desktop and identify object with lowest squared size height times width but also that leads to issues. FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now