Using UI Automation Code in AutoIt

LarsJ · December 22, 2018

This example is about using Microsoft UI Automation APIs in AutoIt. Ie. it's about using interfaces, objects, methods, properties and constants as they are defined directly by Microsoft.

In AutoIt, the definitions are contained in CUIAutomation2.au3 by junkew. The definitions are copied from corresponding C/C++ header files and customized the AutoIt language.

This example is not about the functions in UIAWrappers.au3 by junkew.

The purpose of the example is to show how to use Microsoft UI Automation APIs directly without any intermediate AutoIt functions.

A major advantage of this approach is that the Microsoft documentation directly can be used as AutoIt documentation.

Other major advantages are that there is no need for documenting and maintaining a set of intermediate AutoIt functions. Without intermediate AutoIt functions the code will be very fast. All Microsoft functions can be used this way eg. functions for event handling. New interfaces, objects, methods, properties and constants can easily be added simply by adding the definitions to CUIAutomation2.au3.

Is this approach difficult? Not at all as the examples will show.

The example is inspired by this comment of mLipok. Including documentation and pictures, even a simple Notepad automation can be somewhat comprehensive. Therefore a new example.

First post will be a list of examples. The examples themselves are reviewed in the following posts.

In all examples UIASpy is used to get element information.

Topics

Using UIASpy

UIASpy thread
Automating Notepad. Very detailed example.
Automating Notepad with Sample code - step by step
Automating Notepad with Sample code - all at once
Automating Notepad - Windows XP
Automating Notepad - Classic
Click Save As... issue
Automating Notepad context menu. Event handler example.
Chrome - Clicking an extension. Compact summary example.

Other spy tools

How to topics

If UI Automation or the UIASpy tool is new to you, then you should start reading the
How to topics

Various topics

Console and terminal windows
- Console application (3 posts)
- Terminal program (several posts)

Examples

Threads

UI Automation UDFs contains all include files.
UIASpy - UI Automation Spy Tool is a GUI tool that provides information about windows and controls and their interconnection and provides functionality to generate sample code. UIASpy is essential for creating UI Automation code.
UI Automation Events is about implementing event handlers and includes GUIs to detect events.
IUIAutomation MS framework automate chrome, FF, IE, .... created by junkew August 2013 is the first AutoIt thread on UIA code.

Zip-file
The zip contains source files for all examples.

Note that UI Automation UDFs must be installed in the Includes folder.

You need AutoIt 3.3.12 or later. Tested on Windows XP, Windows 7 and Windows 10.

Comments and questions about using UIA code are welcome. Let me know if there are any issues.

Questions about the functions in UIAWrappers.au3 should be asked in junkew's thread.

UIAExamples.7z

Edited February 9, 2020 by LarsJ
Updates, new zip-file

LarsJ · December 22, 2018

Automating Notepad
This example is a response to a request by mLipok for a simple Notepad automation.

Task: Create a simple script to fill up "Edit1" with HelloWorld, click "Save As...", enter filename and finally save it to disc.

This involves automating Notepad or more generally an Edit control, it involves automating a menu and a Save As dialog.

Note that "Edit1" (from AutoIt Window Info tool) is not a valid control identification in UI Automation code.

Do not open Notepad in advance. This will be done in the code.

Start the code by opening Notepad and create the UI Automation object through IUIAutomation interface.

I almost always create the Desktop element. Note how you first get a pointer to the element (GetRootElement method of $oUIAutomation object), and then creates the object that represents the element through IUIAutomationElement interface. UI Automation elements are nearly always created this way through a pointer.

#include "..\..\Includes\CUIAutomation2.au3"

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 ) 

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

Open Examples\Notepad\Notepad.au3 (empty) in SciTE. Add the code, run the code with F5 and leave Notepad open. Check that SciTE output is OK.

Next step is to get the Notepad window as an UI Automation element. Open UIASpy (included in zip-file, keep UIASpy open throughout the entire example), place mouse cursor over Notepad title bar and press F2:

(You can delete other windows in UIASpy by right-clicking treeview top windows (level one items).)

$UIA_ClassNamePropertyId = "Notepad" is used to create a property condition to identify Notepad.

Note that you can copy this line in UIASpy listview: Select the line in the listview, click "Right pane" item in main menu, click "Copy all or selected items to clipboard", paste the line into your code editor.

Add this code to the existing code:

; --- Notepad window ---

  ConsoleWrite( "--- Notepad window ---" & @CRLF )

  Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )
EndFunc

Note again how we first get a pointer to Notepad (FindFirst method of $oDesktop object, $oDesktop because the Desktop is the parent element of Notepad) and then creates the object. Also note that $oDesktop and $oNotepad are created in exactly the same way.

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.

Now, the actual automation starts: Fill up "Edit1" with HelloWorld.

An automation task is almost always done in two or three steps:
1. Create condition to find element
2. Find the UI Automation element
3. Extract info or perform action

Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included).

To get the Notepad window above we only used two steps.

The first automation task is to perform an action to fill out the Edit control with "HelloWorld". Place mouse cursor in the Edit control and press F1:

(If there are two or more Notepad windows in UIASpy, you can delete the old windows (red) by right-clicking the treeview.)

This time we can use $UIA_AutomationIdPropertyId = "15" to create a property condition to identify the Edit control. Add this code:

; --- Fill Edit element ---

  ConsoleWrite( "--- Fill Edit element ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )
EndFunc

$oNotepad.FindFirst means that the search for the Edit control starts in the Notepad window. $oEdit reprecents the Edit control.

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.

The Edit control must be filled with "HelloWorld". This is an action that has to be performed on the Edit control. The following actions are available:

(Scroll down UIASpy listview to see Control Patterns (actions).)

The interesting action or pattern here is the ValuePattern. The action is created and performed this way:

Local $pValue, $oValue
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld" )
EndFunc

$oEdit.GetCurrentPattern means that a pattern is created that performs actions on the Edit control.

GetCurrentPattern method returns a pointer where from the ValuePattern object ($oValue) can be created. $UIA_ValuePatternId and the ValuePattern interface is defined in CUIAutomation2.au3. The $oValue object represents the ValuePattern interface for the Edit control.

$oValue.SetValue( "HelloWorld" ) sets the text in the Edit control.

An action or pattern is almost always created and performed this way.

Add the code, close the previous Notepad, run the code and leave Notepad open.

Open File menu. Place mouse cursor over the File menu and press F1:

(Scroll down UIASpy listview to see Control Patterns (actions) in bottom of image.)

; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pFile, $oFile
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
  $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
  ConsoleWrite( "$oFile OK" & @CRLF )

  Local $pInvoke, $oInvoke
  $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 100 )
EndFunc

The Invoke action or pattern is used to click and open the File menu. Note the similarity between the code for InvokePattern and ValuePattern above.

When a menu, dialog box or child window is opened it need some time to open. You must add a Sleep statement.

Add the code, change "File" to the corresponding word in your own language (copy the word from the listview), close the previous Notepad (don't save), run the code and leave Notepad open.

Click "Save As..." menu. If the File menu that was opened above is closed, then open it manually with a mouse click. Place mouse cursor over the "Save As..." menu and press F1:

; --- Click "Save As..." menu ---

  ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF )

  ; Reuse $pCondition1 above

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAs, $oSaveAs
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs )
  $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF )
  ConsoleWrite( "$oSaveAs OK" & @CRLF )

  $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )
EndFunc

Note how similar the "Open File menu" code and the "Click Save As... menu" code is. Note also the long Sleep because it's a "Save As" dialog with a large number of elements.

Add the code, close the previous Notepad (don't save), run the code and leave Notepad and Save As open.

Save As window. Place mouse cursor over Save As title bar and press F2:

; --- Save As window ---

  ConsoleWrite( "--- Save As window ---" & @CRLF )

  ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )
  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )
EndFunc

The rest of the automation tasks is about the Save As window. Therefore we want to use the Save As window as the starting point. Not Notepad.

Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open.

Set File name. Place mouse cursor over File name Edit control and press F1:

; --- Set File name ---

  ConsoleWrite( "--- Set File name ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld.txt" )
EndFunc

$oSaveAsWin.FindFirst searches in the Save As window. $oEdit is now the File name Edit control in the Save As window. The file name is set to "HelloWorld.txt".

Add the code, close the previous Save As and Notepad (don't save), run the code and leave Notepad and Save As open.

Click Save button. Place mouse cursor over Save button and press F1:

; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSave, $oSave
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave )
  $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF )
  ConsoleWrite( "$oSave OK" & @CRLF )

  $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

Add the code, close the previous Save As and Notepad (don't save) and run the code.

All code (Examples\Notepad\NotepadAll.au3):

#include "..\..\Includes\CUIAutomation2.au3"

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Notepad window ---

  ConsoleWrite( "--- Notepad window ---" & @CRLF )

  Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit element ---

  ConsoleWrite( "--- Fill Edit element ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValue, $oValue
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld" )

  ; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pFile, $oFile
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
  $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
  ConsoleWrite( "$oFile OK" & @CRLF )

  Local $pInvoke, $oInvoke
  $oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 100 )

  ; --- Click "Save As..." menu ---

  ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF )

  ; Reuse $pCondition1 above

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAs, $oSaveAs
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs )
  $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF )
  ConsoleWrite( "$oSaveAs OK" & @CRLF )

  $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )

  ; --- Save As window ---

  ConsoleWrite( "--- Save As window ---" & @CRLF )

  ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )
  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )

  ; --- Set File name ---

  ConsoleWrite( "--- Set File name ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld.txt" )

  ; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSave, $oSave
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave )
  $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF )
  ConsoleWrite( "$oSave OK" & @CRLF )

  $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Notepad window ---
$pCondition OK
$oNotepad OK
--- Fill Edit element ---
$pCondition OK
$oEdit OK
$oValue OK
--- Open File menu ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oFile OK
$oInvoke OK
--- Click "Save As..." menu ---
$pCondition2 OK
$pCondition OK
$oSaveAs OK
$oInvoke OK
--- Save As window ---
$pCondition OK
$oSaveAsWin OK
--- Set File name ---
$pCondition OK
$oEdit OK
$oValue OK
--- Click Save button ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oSave OK
$oInvoke OK

Maybe somebody can show how to implement such a Notepad automation with the functions in UIAWrappers.au3 and with classic code.

Summary
The two most important objects in UI Automation code are the $oUIAutomation and $oUIElement objects created from IUIAutomation and IUIAutomationElement interfaces.

An UI Automation element object that represents a window or control is always created with the IUIAutomationElement interface through a pointer to the element object:

$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
$oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
$oEdit    = ObjCreateInterface( $pEdit,    $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

An UI Automation task is almost always done in two or three steps:
1. Create condition to find element
2. Find the UI Automation element
3. Extract info or perform action

Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included).

UI Automation actions (patterns) are performed through pattern interfaces, objects and methods. The code is nearly always implemented this way:

Fill an Edit control:

Local $pValue, $oValue
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
$oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
ConsoleWrite( "$oValue OK" & @CRLF )
$oValue.SetValue( "HelloWorld" )

$oEdit is an Edit control. The $oValue object represents the ValuePattern interface for the Edit control. $oValue.SetValue() sets the text in the Edit control.

Click a menu item:

Local $pInvoke, $oInvoke
$oFile.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
$oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
ConsoleWrite( "$oInvoke OK" & @CRLF )
$oInvoke.Invoke()
Sleep( 100 )

$oFile is a File menu. The $oInvoke object represents the InvokePattern interface for the File menu. $oInvoke.Invoke() clicks the File menu. Sleep( 100 ) gives the File menu time to open.

Click a button control:

$oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
$oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
ConsoleWrite( "$oInvoke OK" & @CRLF )
$oInvoke.Invoke()

$oSave is a Save button. The $oInvoke object represents the InvokePattern interface for the Save button. $oInvoke.Invoke() clicks the Save button.

When writing UI Automation code, it's easiest to write the code in small steps and test the code after each step. Error management based on checking objects with IsObj() and ConsoleWrite() is required and sufficient. It's very rare necessary to use an actual object (COM) error handler. Too small Sleep times after a menu, dialog or window is opened is a common error.

Edited December 22, 2018 by LarsJ

InnI · December 22, 2018

@LarsJ

Why don't you create a condition object from a pointer and use the pointer directly?

Local $pCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer
Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) ; use pointer

What's the difference between your code and this?

Local $pCondition, $oCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer
$oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition ) ; create object
Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad ) ; use object

mLipok · December 22, 2018

This is what I was looking for.

BIG THANKS @LarsJ

Edited December 22, 2018 by mLipok

junkew · December 22, 2018

Nice detailed example and yes uiautomation of ms is in general fast when you can do exact matching.

Similar examples are given in examples.zip of uiautomation thread. Main reason for the uiawrappers is to have more flexibility on

Index, indexrelative
Small wrappers around pointer and objcreateinterface logic
Named properties and easy combining them
At some cost of speed
Setvalue, settext interface on many objects not working
Invoke sometimes is a click and sometimes a setfocus.
Match on regular expressions

LarsJ · December 23, 2018

InnI, Because these 2 ConsoleWrites prints the same value

ConsoleWrite( "Ptr( $pCondition ) = " & Ptr( $pCondition ) & @CRLF )
ConsoleWrite( "Ptr( $oCondition() ) = " & Ptr( $oCondition() ) & @CRLF )

then I don't think there is much difference between these lines:

$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad )
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition(), $pNotepad )

But you are using an additional line to create the $oCondition object. Since you are not calling any methods of the object, it seems to be an unnecessary additional line.

junkew, There are none of these points that cannot be implemented even if one uses a direct approach as demonstrated here. Using the Microsoft code in a direct way is simply not a limitation. You can make an example or code snippet covering each situation. I have already made a number of examples based on this technique, most of them in your own thread. There is nothing that can't be done.

Edited December 23, 2018 by LarsJ

Earthshine · December 23, 2018

You guys rule. This makes my Christmas. Thanks LarsJ and junkew both for making my automated life better!

InnI · December 23, 2018

1 hour ago, LarsJ said:

it seems to be an unnecessary additional line

Could you please explain, why there is a constant increase in memory usage in this code?

#include "CUIAutomation2.au3"

HotKeySet("{esc}", "Quit")
Func Quit()
  Exit
EndFunc

Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
Global $pCondition, $pDesktop, $oDesktop, $pProgman, $oProgman
$oUIAutomation.GetRootElement( $pDesktop )
$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

While 1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition )
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pProgman )
  $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj($oProgman) Then Exit
WEnd

But in this code, memory usage increases slightly and remains constant

#include "CUIAutomation2.au3"

HotKeySet("{esc}", "Quit")
Func Quit()
  Exit
EndFunc

Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
Global $pCondition, $oCondition, $pDesktop, $oDesktop, $pProgman, $oProgman
$oUIAutomation.GetRootElement( $pDesktop )
$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

While 1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition )
  $oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition )
  $oDesktop.FindFirst( $TreeScope_Children, $oCondition, $pProgman )
  $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj($oProgman) Then Exit
WEnd

It looks like a memory leak occurs when to use the pointer directly.

argumentum · December 23, 2018

When I investigate SciTE, ( with any UI tool ) it crashes when I try to "F5" to run a script. So I guess SciTE is not UIA friendly.
Question: is there a way to ask an executable if is UIA friendly ?

junkew · December 23, 2018

What do you mean with UIA friendly?

Automating SCITE works fine for me (copy, pasted with simplespy merged and some small fixes on patterns for recognition)

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=SciTE;controltype:=UIA_WindowControlTypeId;class:=SciTEWindow", $treescope_children)
;~ _UIA_Action($oP1,"setfocus")
_UIA_action($oP1,"highlight")

Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Toepassing;controltype:=UIA_MenuBarControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
;~ $oUIElement=_UIA_getObjectByFindAll("Tools.mainwindow", "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Tools;controltype:=UIA_MenuControlTypeId;class:=#32768", $treescope_children)
_UIA_Action($oP0,"setfocus")
_UIA_Action($oP0,"highlight")

;~ First find the object in the parent before you can do something
;~ $oUIElement=_UIA_getObjectByFindAll("Go  F5.mainwindow", "title:=Go  F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Go  F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"highlight")
;~ _UIA_action($oUIElement,"click")

argumentum · December 24, 2018

On 12/23/2018 at 3:33 PM, junkew said:

What do you mean with UIA friendly? Automating SCITE works fine for me

hmmm, I'm trying to replicate the crashes I experienced before, ...but it wont crash. Tho my PC rebooted due to windows update. Maybe all it needed was a reboot.
Sowy, disregard my question and thanks for the answer =)

PS: image.png.647be33c8def5390045aed7ecb203647.png

.~~..I'll look into it.~~

PS2: in SciTE v3.6.6 there is no crash but in SciTE v4.1.0 it crashes with the above Pic.
Be aware that I am not talking about running code. The crash is after I use, any tool really, to highlight the ID/Text/etc., and press F5 to run any code from the editor, is that the editor crashes.

PS3: use "SampleControls.au3" from your hard drive to replicate.

Edited December 25, 2018 by argumentum
deep, ( or not ), investigation result

LarsJ · December 24, 2018

InnI, I can't explain that. But in relation to UI Automation code, it is not really a problem because we're dealing with a very limited number of objects.

junkew · January 1, 2019

Based on something you wrote in 2014 in UIAutomation thread I am curious on if this still is a valid statement? why?
Any example where this has to be done on a propertycondition?

Quote
If you are running a 32 bit script, you must change the CreatePropertyCondition description in this way:
"CreatePropertyCondition hresult(int;uint64;uint64;ptr*);"
If you are running a 64 bit script, you must change the CreatePropertyCondition description in this way:
"CreatePropertyCondition hresult(int;ptr;ptr*);"
You need this description for an advanced data type like a structure (created with DllStructCreate). For simple data types like numbers and strings, you still use the old description with a variant. The reason for two new defitions are caused by differences in parameter transfer on 32 and 64 bit systems.

I am working on (MIDL to AU3 converter) instead of CUIAutomation2.au3 we will have UIAutomationClient.au3 based on the IDL UIAutomationClient.IDL and then I will generate

;~ HRESULT CreatePropertyCondition ([in] PROPERTYID propertyId, [in] VARIANT value, [out, retval] IUIAutomationCondition ** newCondition );
 "CreatePropertyCondition hresult(struct;variant;ptr*);"

Edited January 1, 2019 by junkew

LarsJ · January 2, 2019

This is still valid. It must be taken into account in situations where a method takes an array as input parameter and the array is not an array of variants.

If you search for VT_ARRAY in this list you'll get 13 matches. When you use any of these PropertyIds as input parameters you have to handle the arrays properly.

The problem arises because the internal AutoIt COM conversions that converts native AutoIt variables and arrays to COM compatible data types always convert a native AutoIt array into a safearray of variants.

But in these 13 situations the arrays are safearrays of integers, doubles and IUnknown pointers and thus not safearrays of variants. Here we have to handle the COM conversions ourselves.

When a safearray is passed as a parameter to a COM method it's usually passed as a variant ($tagVariant structure) where the variant data field contains a pointer to the safearray.

In 32 bit code parameters are passed through the stack. In 32 bit code a variant takes up 16 bytes. The purpose of "uint64;uint64;" is to pass the 16 bytes variant structure onto the stack wherefrom it's read by the internal method code.

In 64 bit code the first 4 parameters are passed through CPU registers. If one of the first 4 parameters is a structure and the structure isn't larger than 8 bytes (64 bit) it's passed directly through the CPU register. If it's larger than 8 bytes a pointer to the structure is passed through the CPU register.

In 64 bit code a variant takes up 24 bytes. The purpose of "ptr;" is to pass a pointer to the variant structure through a CPU register wherefrom the pointer and then the variant structure is read by the internal method code.

If you read the posts around the quoted post, you'll see a working example.

junkew, Let's make a little deal. I'll code a function to handle arrays as input parameters and post it here. Then you can copy the function directly into your new UDF. In return, I'll use your UDF in my code.

junkew · January 2, 2019

thx for the explanation.

Deal is great

The new UIA UDF will be based on

the UIA IDL files of Win10 SDK, (So I think 2-3 files) generated by the converter
https://www.autoitscript.com/forum/topic/197177-idl-to-au3-generator/
Little in doubt if the generator is just good enough and some manual changes to be done afterward on generated au3 files (alternative is to build IDL lexer/parser but more work or use OAIdl.Idl to make an .au3 file and use the typelib info to generate then the au3 files)
UIAWrappers will be then based on
- definitions of step 1
- improved on finding on single propertyconditions (input you gave in this thread and UIA thread) where no regex finding is needed
Extended simplespy with some addition coding examples when spied on object
fixed/cleaned examples

LarsJ · January 25, 2019

Automating Notepad with Sample code - step by step

This is a repetition of the Notepad example above. But this time, the possibilities for creating Sample code in UIASpy will be exploited.

Only the first part of the example will be reviewed once more in detail. But the pictures will not be repeated. So please open the previous example in a new tab in the browser.

Open "Examples\Notepad\Windows 7, 10\2) Sample code - step by step\Notepad.au3" (empty) in SciTE. The file is included in the zip-file in bottom of first post.

Open UIASpy (also included in zip-file) and keep UIASpy open throughout the entire example. Open Left pane main menu and click Delete top windows to delete all treeview windows except Desktop.

First step is to add initial code to create the UI Automation object and the Desktop element. Open Sample code main menu in UIASpy and click Initial code.

Open the menu once more and click Corrections to get an idea of the corrections that have to be made to this generic code.

Paste the code from clipboard into SciTE (Shift+Insert). The code should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Correct the path to CUIAutomation2.au3. Add code to open Notepad. Delete correction comments. Now the code should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

Run the code with F5 and leave Notepad open. Check that SciTE output is OK.

Next step is to get the Notepad window as an UI Automation element. Place mouse cursor over Notepad title bar and press F2.

Select the row in the Notepad detail information listview page in UIASpy that matches the row with the red arrow in the first picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code.

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pWindow, $oWindow
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Replace "window/control" with "Notepad window". Replace "Window" with "Notepad". Delete correction comments. Move code into function:

; --- Find Notepad window ---

ConsoleWrite( "--- Find Notepad window ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
$oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
ConsoleWrite( "$oNotepad OK" & @CRLF )

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.

Third step is to fill out the Edit control with "HelloWorld".

An UI Automation task is almost always done in two or three steps:
1. Create condition to find element
2. Find the UI Automation element
3. Extract info or perform action

Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included).

To get the Notepad window above we only used two steps.

Place mouse cursor in the Edit control in Notepad and press F1.

Select the row in the Edit detail information listview page in UIASpy that matches the row with the red arrow in the second picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code.

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Replace "Find window/control" with "Fill Edit control". Delete the line "Local $pCondition0". Replace "$oParent" with "$oNotepad". Delete correction comments. Move code into function:

; --- Fill Edit control ---

ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pEdit, $oEdit
$oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.

Place mouse cursor in the Edit control in Notepad and press F1. If there is a red (invalid) Notepad window in the treeview then delete it (Right-click the red element and click Delete element and all childs).

Scroll down to the Control Patterns (element actions) section in the Edit detail information listview page in UIASpy.

Select the row "$UIA_IsValuePatternAvailablePropertyId" that matches the row with the red arrow in the third picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Click the Edit item in the treeview to return to the Edit detail information listview page. Scroll down to the bottom of the listview page. The Value Pattern Methods subsection should be visible.

Select the row with the SetValue(bstr) method in second column:

Right-click and click Create sample code (don't clear the code this time).

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )

; --- Value Pattern (action) Methods ---

ConsoleWrite( "--- Value Pattern (action) Methods ---" & @CRLF )

$oValuePattern.SetValue(bstr)

Delete the two comments, two ConsoleWrites and empty lines. Replace "bstr" with "HelloWorld". Move code into function:

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue( "HelloWorld" )

Close the previous Notepad, run the code, check that SciTE output is OK, check "HelloWorld" in Notepad Edit control.

All code after the third step:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find Notepad window ---

  ConsoleWrite( "--- Find Notepad window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit control ---

  ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValuePattern, $oValuePattern
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld" )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find Notepad window ---
$pCondition0 OK
$oNotepad OK
--- Fill Edit control ---
$pCondition0 OK
$oEdit OK
$oValuePattern OK

Edited January 25, 2019 by LarsJ

LarsJ · January 25, 2019

Automating Notepad with Sample code - all at once

In an automation task like the Notepad example above, it's much faster to create all sample code at once than it is to create the sample code in a step-by-step procedure.

For pattern methods, it's also faster to create sample code by selecting the method through the Pattern... item in the Sample code menu in UIASpy than it is to create pattern method sample code through the detail information listview page.

But to use the pattern listview page you need to know which pattern to use. The advantage of using the detail information listview page is that it only shows the patterns and pattern methods that are applicable to the specific control.

These are all the steps in the Notepad example:

Add initial code
Open Notepad
- Sleep( 1000 )
Get Notepad window
Fill Edit control
- Identify Edit control
- Execute SetValue pattern method
Open File menu
- Identify File menu
- Execute Invoke pattern method
- Sleep( 100 )
Click Save As... menu item
- Identify Save As... menu item
- Execute Invoke pattern method
- Sleep( 1000 )
Get Save As window
Fill Edit control (File name)
- Identify Edit control
- Execute SetValue pattern method
Click Save button
- Identify Save button
- Execute Invoke pattern method
Corrections

And this is sample code for all the steps in the entire Notepad example.

Examples\Notepad\Windows 7, 10\3) Sample code - all at once\1) Before corrections.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pWindow, $oWindow
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
ConsoleWrite( "$pCondition1 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue(bstr)

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition2, $pCondition3, $pAndCondition3
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 )
$oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

Sleep( 1000 )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition4, $pCondition5, $pAndCondition5
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 )
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 )
$oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 )
If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition5 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

Sleep( 1000 )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition6
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 )
If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF )
ConsoleWrite( "$pCondition6 OK" & @CRLF )

Local $pWindow, $oWindow
$oParent.FindFirst( $TreeScope_Descendants, $pCondition6, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition7
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 )
If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF )
ConsoleWrite( "$pCondition7 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue(bstr)

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition8, $pCondition9, $pAndCondition9
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 )
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 )
$oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 )
If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition9 OK" & @CRLF )

Local $pButton, $oButton
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton )
$oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
ConsoleWrite( "$oButton OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

The code isn't immediately executable. It's necessary to make some corrections. Most corrections are listed at the bottom of the code as comments.

Note that all condition variables are created properly. They need no corrections.

This is the code after corrections.

Examples\Notepad\Windows 7, 10\3) Sample code - all at once\2) After corrections.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Get Notepad window ---

  ConsoleWrite( "--- Get Notepad window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit control ---

  ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValuePattern, $oValuePattern
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld" )

  ; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition2, $pCondition3, $pAndCondition3
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 ) ; <<<<<<<<<<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  Local $pMenuItem, $oMenuItem
  $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  Local $pInvokePattern, $oInvokePattern
  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  Sleep( 100 )

  ; --- Click Save As... menu item ---

  ConsoleWrite( "--- Click Save As... menu item ---" & @CRLF )

  Local $pCondition4, $pCondition5, $pAndCondition5
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 )
  $oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 )
  If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition5 OK" & @CRLF )

  $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  Sleep( 1000 )

  ; --- Get Save As window ---

  ConsoleWrite( "--- Get Save As window ---" & @CRLF )

  Local $pCondition6
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 )
  If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF )
  ConsoleWrite( "$pCondition6 OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition6, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )
  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )

  ; --- Fill Edit control (File name) ---

  ConsoleWrite( "--- Fill Edit control (File name) ---" & @CRLF )

  Local $pCondition7
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 )
  If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF )
  ConsoleWrite( "$pCondition7 OK" & @CRLF )

  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld.txt" )

  ; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  Local $pCondition8, $pCondition9, $pAndCondition9
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 )
  $oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 )
  If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition9 OK" & @CRLF )

  Local $pButton, $oButton
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
EndFunc

When executing the code, it should perform all Notepad automation tasks and generate this output in SciTE.

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Get Notepad window ---
$pCondition0 OK
$oNotepad OK
--- Fill Edit control ---
$pCondition1 OK
$oEdit OK
$oValuePattern OK
--- Open File menu ---
$pAndCondition3 OK
$oMenuItem OK
$oInvokePattern OK
--- Click Save As... menu item ---
$pAndCondition5 OK
$oMenuItem OK
$oInvokePattern OK
--- Get Save As window ---
$pCondition6 OK
$oSaveAsWin OK
--- Fill Edit control (File name) ---
$pCondition7 OK
$oEdit OK
$oValuePattern OK
--- Click Save button ---
$pAndCondition9 OK
$oButton OK
$oInvokePattern OK

Edited January 26, 2019 by LarsJ

LarsJ · January 26, 2019

Automating Notepad - Windows XP
This is a repetition of the Notepad example above. But this time under Windows XP.

I'll not review the whole example once more but only focus on the challenges of Windows XP. These challenges are about not all pattern methods working under Windows XP.

There are two cases where the pattern methods does not work.

The first case is to open the File menu by clicking the menu item. Here, the Invoke method does not work. The only thing you can do is use the bounding rectangle to position the mouse over the File menu and perform a mouse click:

; --- Open File menu ---

ConsoleWrite( "--- Open File menu ---" & @CRLF )

Local $pCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
ConsoleWrite( "$pCondition1 OK" & @CRLF )

Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
ConsoleWrite( "$pCondition2 OK" & @CRLF )

; And condition
$oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
ConsoleWrite( "$pCondition OK" & @CRLF )

Local $pFile, $oFile
$oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
$oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
ConsoleWrite( "$oFile OK" & @CRLF )

; Rectangle
Local $aRect ; l, t, w, h
$oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
ConsoleWrite( "$aRect OK" & @CRLF )

; Click File menu
MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
Sleep( 100 )

The second case is to fill out the File name (an Edit control) in the "Save As ..." dialog. The SetValue pattern method does not work in this Edit control. The only thing that works is the Send command.

Note that the File name control gets focus when the "Save As ..." dialog opens and that the text in Edit control is selected:

; --- Set File name ---

ConsoleWrite( "--- Set File name ---" & @CRLF )

ConsoleWrite( "Send()" & @CRLF )
Send( "HelloWorld.txt", 1 )

All Windows XP code.

Examples\Notepad\Windows XP\1) Create code\NotepadAll.au3:

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Notepad window ---

  ConsoleWrite( "--- Notepad window ---" & @CRLF )

  Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit element ---

  ConsoleWrite( "--- Fill Edit element ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValue, $oValue
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld" )

  ; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pFile, $oFile
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
  $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
  ConsoleWrite( "$oFile OK" & @CRLF )

  ; Rectangle
  Local $aRect ; l, t, w, h
  $oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
  If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
  ConsoleWrite( "$aRect OK" & @CRLF )

  ; Click File menu
  MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
  Sleep( 100 )

  ; --- Click "Save As..." menu ---

  ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF )

  ; Reuse $pCondition1 above

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAs, $oSaveAs
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs )
  $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF )
  ConsoleWrite( "$oSaveAs OK" & @CRLF )

  Local $pInvoke, $oInvoke
  $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )

  ; --- Save As window ---

  ConsoleWrite( "--- Save As window ---" & @CRLF )

  ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin, $hSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSaveAsWin.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hSaveAsWin )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )

  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )

  ; --- Set File name ---

  ConsoleWrite( "--- Set File name ---" & @CRLF )

  ConsoleWrite( "Send()" & @CRLF )
  Send( "HelloWorld.txt", 1 )

  ; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSave, $oSave
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave )
  $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF )
  ConsoleWrite( "$oSave OK" & @CRLF )

  $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Notepad window ---
$pCondition OK
$oNotepad OK
--- Fill Edit element ---
$pCondition OK
$oEdit OK
$oValue OK
--- Open File menu ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oFile OK
$aRect OK
--- Click "Save As..." menu ---
$pCondition2 OK
$pCondition OK
$oSaveAs OK
$oInvoke OK
--- Save As window ---
$pCondition OK
$oSaveAsWin OK
--- Set File name ---
Send()
--- Click Save button ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oSave OK
$oInvoke OK

Edited January 26, 2019 by LarsJ

LarsJ · January 27, 2019

Chrome - Clicking an extension
A day ago, there was a question of clicking a Chrome extension.

This is a demonstration of how it can be done with UIASpy. Here the "Blank New Tab Page" button is clicked. Chrome is already open and displaying the AutoIt start page.

Because this is about the outer elements of Chrome it's important that accessibility is not enabled.

Open UIASpy, turn off Only visible elements in the Left pane menu, place the mouse cursor over the "Blank New Tab Page" button and press F1:

The butten is located below a pane control. That's why Only visible elements has to be turned off. This is also the reason why the button cannot be identified with the AutoIt Window Info tool. And junkew's simplespy.au3 will identify the pane control instead of the button.

So we are not at all interested in the pane control. We are interested in the Chrome top window:

Open Sample code main menu, click Initial code, click the selected treeview item to return to the detail info page, right click the selected row and click Create sample code.

Then we are interested in the "Blank New Tab Page" button:

Repeat this for each group of selected row(s): Right click a selected row and click Create sample code.

Click the "Blank New Tab Page" button in Chrome to open a menu (see the picture above). Place the mouse cursor over the "Blank New Tab Page" menu item and press F1. The menu is closed when UIASpy gets focus. That's why the treeview items are red:

Repeat this for each group of selected row(s): Right click a selected row and click Create sample code.

Because I've tested the code, I know that the Invoke method doesn't work for the "Blank New Tab Page" menu item. We have to use a mouse click:

Right click the selected row and click Create sample code.

Open Sample code main menu, click Corrections. Paste the code into an editor.

All sample code (Examples\Chrome\Click extension button\1) Before corrections.au3) should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pPane, $oPane
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pPane )
$oPane = ObjCreateInterface( $pPane, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oPane ) Then Return ConsoleWrite( "$oPane ERR" & @CRLF )
ConsoleWrite( "$oPane OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition1, $pCondition2, $pAndCondition2
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 )
$oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 )
If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition2 OK" & @CRLF )

Local $pButton, $oButton
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton )
$oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
ConsoleWrite( "$oButton OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )

; --- Invoke Pattern (action) Methods ---

ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

$oInvokePattern.Invoke()

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition3, $pCondition4, $pAndCondition4
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 )
$oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 )
If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

; --- Element Properties (information) ---

ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )

Local $vValue
$oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $vValue )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Code after corrections (Examples\Chrome\Click extension button\2) After corrections.au3):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find Chrome window ---

  ConsoleWrite( "--- Find Chrome window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pChrome, $oChrome
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pChrome )
  $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )

  ; --- Find "Blank New Tab Page" button ---

  ConsoleWrite( "--- Find ""Blank New Tab Page"" button ---" & @CRLF )

  Local $pCondition1, $pCondition2, $pAndCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 )
  If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition2 OK" & @CRLF )

  Local $pButton, $oButton
  $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern, $oInvokePattern
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern.Invoke()
  Sleep( 100 )

  ; --- Find "Blank New Tab Page" menu item ---

  ConsoleWrite( "--- Find ""Blank New Tab Page"" menu item ---" & @CRLF )

  Local $pCondition3, $pCondition4, $pAndCondition4
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  Local $pMenuItem, $oMenuItem
  $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  #cs
  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  #ce

  ; Rectangle
  Local $aRect ; l, t, w, h
  $oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
  If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
  ConsoleWrite( "$aRect OK" & @CRLF )

  ; Click menu item
  MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find Chrome window ---
$pCondition0 OK
$oChrome OK
--- Find "Blank New Tab Page" button ---
$pAndCondition2 OK
$oButton OK
--- Invoke Pattern (action) Object ---
$oInvokePattern OK
--- Invoke Pattern (action) Methods ---
--- Find "Blank New Tab Page" menu item ---
$pAndCondition4 OK
$oMenuItem OK
$aRect OK

New zip-file at bottom of first post.

Edited January 31, 2019 by LarsJ
A few text updates

junkew · January 28, 2019

Nice examples. Simplespy just uses elementfrompoint and you could argue pane as child from button is not a logical gui tree. Similar behavior you can see when automating ms access forms. An improved spy i tried scan hierarchy elementfrompoint up to desktop and identify object with lowest squared size height times width but also that leads to issues.

Using UI Automation Code in AutoIt

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

LarsJ

LarsJ

LarsJ

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members