Using UI Automation Code in AutoIt

seadoggie01 · November 6, 2019

I couldn't identify the dropdown separately from the rest of the MenuItem and UIA Spy didn't seem to see it either. I tried navigating with the function keys, but even then it didn't see it. I'm not sure where to go from here... should I just click it?

Edit: So there's a setting I can change to turn the menu item into a button... but then I need to right click it to expand the menu, is that possible instead?

Edit2: Found it, UIA_MouseClick

Edited November 6, 2019 by seadoggie01

junkew · November 6, 2019

Uiaclick determines the rectangle and mousemove puts pointer in center before sending mouseclick. This way you prevent frezing your script.

seadoggie01 · November 6, 2019

Thanks junkew! UIA_MouseClick does the same thing (and it hides the mouse while clicking, which I never thought of before)

LarsJ · November 21, 2019

UIA updates
The examples here show how Sample code (created through the Detail info listview page and the Sample code menu in UIASpy) looks when the code is generated under different Windows modes (Options | Windows | Mode menu in UIASpy). Note the version differences in CLSID_CUIAutomation as well as the main $oUIAutomation and $oUIElement objects.

When you select Windows mode through the Options | Windows | Mode menu, UIASpy will generate correct Sample code for this Windows version and use the correct CLSID_CUIAutomation version and the correct versions of the main $oUIAutomation and $oUIElement objects.

Code is stored in Examples\7) UIA updates\Notepad\ folder. The examples show code generated under Windows 7, Windows 8, Windows 8.1, Windows 10 First (1507) and Windows 10 Last (1809) modes. The code boxes below show code for Windows 7 and Windows 10 Last modes.

Windows 7 mode (Notepad-W7.au3):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pWindow1 ) ; $TreeScope_Children to find application top window
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )       ; (much faster)
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 ) ; $TreeScope_Descendants to find window controls
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )
EndFunc

The code can be executed under Windows XP, Windows Vista, Windows 7 and later versions.

Windows 10 Last (1809) mode (Notepad-W10Last.au3):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation8, $sIID_IUIAutomation6, $dtag_IUIAutomation6 )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement9, $dtag_IUIAutomationElement9 )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement9, $dtag_IUIAutomationElement9 )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement9, $dtag_IUIAutomationElement9 )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )
EndFunc

The code can be executed under Windows 10 version 1809 and later versions.

Output in SciTE console for all examples:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oWindow1 OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK

New zip-file at bottom of first post.

LarsJ · January 4, 2020

Patterns (actions), 16 - 17
Control patterns are objects that can be used to perform actions on an UI element (window/control). The actual actions are performed by executing the methods of the pattern objects.

Control types that are supported by specific control patterns are listed in Control Types and Their Supported Control Patterns.

How to create pattern objects and execute pattern methods through UIASpy is described in How to topics 12 and 14.

The examples shows how to use control patterns. The examples are tested on Windows 10 and Windows 7.

In many examples you need to update eg. control names to your own names. There may also be other values to be updated. Lines to be updated are marked with

; <<<<<<<<<<<<<<<<<<<<

Use UIASpy to check and copy the values.

All code is stored in Examples\4) Patterns (actions)\<Pattern>\ folders.

Patterns (actions):

Section 1 - 12: Examples based on File Explorer
Section 13 - 15: Virtual listview item patterns
Section 16 - 17: Other control pattern objects

File Explorer picture

Window patterns
Window Control Pattern
Transform Control Pattern

Treeview patterns
ExpandCollapse Control Pattern
ScrollItem Control Pattern
Scroll Control Pattern
Does not work on Windows 7 ($oScrollPattern1 ERR)

Listview patterns
Grid Control Pattern
Table Control Pattern
Invoke Control Pattern
MultipleView Control Pattern
SelectionItem Control Pattern
Selection Control Pattern

Virtual listview item patterns
File Explorer picture
ItemContainer Control Pattern
VirtualizedItem Control Pattern

Other control pattern objects
Value Control Pattern
RangeValue Control Pattern (SetValue())

16. Value Control Pattern
From Microsoft documentation:
The Value control pattern is used to support controls that have an intrinsic value not spanning a range and that can be represented as a string.

An Edit control is an obvious example.

Preparation
Open an empty WordPad window. Open UIASpy, place the mouse over the Edit control, press F1.

Code
The code in Value.au3 shows how to set a text value in the Edit control (Value-a.au3 contains code directly from UIASpy):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "WordPadClass", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "RICHEDIT50W", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )

  ; --- Value Pattern (action) Object ---

  ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

  Local $pValuePattern1, $oValuePattern1
  $oEdit1.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern1 )
  $oValuePattern1 = ObjCreateInterface( $pValuePattern1, $sIID_IUIAutomationValuePattern, $dtag_IUIAutomationValuePattern )
  If Not IsObj( $oValuePattern1 ) Then Return ConsoleWrite( "$oValuePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern1 OK" & @CRLF )

  ; --- Value Pattern (action) Methods ---

  ConsoleWrite( "--- Value Pattern (action) Methods ---" & @CRLF )

  $oValuePattern1.SetValue( "This is a test" )
  ConsoleWrite( "$oValuePattern1.SetValue()" & @CRLF )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oWindow1 OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK
--- Value Pattern (action) Object ---
$oValuePattern1 OK
--- Value Pattern (action) Methods ---
$oValuePattern1.SetValue()

Remarks
Note that there are several changes to the new code compared to the old code in the previous examples from March 2019.

17. RangeValue Control Pattern
From Microsoft documentation:
The RangeValue control pattern is used to support controls that can be set to a value within a range.

Eg. a Slider control.

Preparation
Open an empty WordPad window. Enter a short text in the Edit control. The Slider control in the lower right corner should display the value 100%. Open UIASpy, place the mouse over the Slider control, press F2.

Code
RangeValue.au3 shows how to set the Slider value to zoom in to 300% (RangeValue-a.au3 contains code directly from UIASpy):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "WordPadClass", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_SliderControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pSlider1, $oSlider1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition1, $pSlider1 )
  $oSlider1 = ObjCreateInterface( $pSlider1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oSlider1 ) Then Return ConsoleWrite( "$oSlider1 ERR" & @CRLF )
  ConsoleWrite( "$oSlider1 OK" & @CRLF )

  ; --- RangeValue Pattern (action) Object ---

  ConsoleWrite( "--- RangeValue Pattern (action) Object ---" & @CRLF )

  Local $pRangeValuePattern1, $oRangeValuePattern1
  $oSlider1.GetCurrentPattern( $UIA_RangeValuePatternId, $pRangeValuePattern1 )
  $oRangeValuePattern1 = ObjCreateInterface( $pRangeValuePattern1, $sIID_IUIAutomationRangeValuePattern, $dtag_IUIAutomationRangeValuePattern )
  If Not IsObj( $oRangeValuePattern1 ) Then Return ConsoleWrite( "$oRangeValuePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oRangeValuePattern1 OK" & @CRLF )

  ; --- RangeValue Pattern (action) Methods ---

  ConsoleWrite( "--- RangeValue Pattern (action) Methods ---" & @CRLF )

  $oRangeValuePattern1.SetValue(75)
  ConsoleWrite( "$oRangeValuePattern1.SetValue()" & @CRLF )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oWindow1 OK
--- Find window/control ---
$pCondition1 OK
$oSlider1 OK
--- RangeValue Pattern (action) Object ---
$oRangeValuePattern1 OK
--- RangeValue Pattern (action) Methods ---
$oRangeValuePattern1.SetValue()

Edited February 9, 2020 by LarsJ
Text updates

LarsJ · February 9, 2020

UIA events
The "FocusChangedEventHandler example" section of the first post in the "UI Automation Events" thread also contains brief descriptions of examples demonstrating the use of AutomationEventHandler, PropertyChangedEventHandler, and StructureChangedEventHandler. These four event handlers are available in all Windows versions from Windows XP to Windows 10.

A later post describes an example demonstrating the use of the NotificationEventHandler added in Windows 10 1709.

tsrkumar · July 21, 2020

Larsj

I am getting familiar with AutoiT and am using the UIASpy automation tool in my current automation effort that deals with Rocket Shuttle.

I am trying to get the child Window / Control from the desktop object using the processid.

However, I am running into this error.

$oUIAutomation.CreatePropertyCondition($UIA_ProcessIdPropertyId, $Pid, $pCondition0) gives back a HRESULT of -2147024809 which is invalid args.

The $Pid is the result of the RUN command. I understand that the 2nd argument to the function is a Variant. I am at loss to see how I can convert the $pid to a VT_I4 variant.

The code works fine if I pass a hardcoded process id integer value.

Thanks.

LarsJ · July 21, 2020

The problem is that Run() returns the PID as a variant of type double (VT_R8). You can find out with the VarGetType() function. All AutoIt variables are variants. But you can easily convert this double to an integer with the Int() function.

tsrkumar · July 21, 2020

Thanks Larsj! that did the trick!

Appreciate the insight into AutoIt variables and the way to obtain their type.

tsrkumar · July 21, 2020

7 hours ago, LarsJ said:

The problem is that Run() returns the PID as a variant of type double (VT_R8). You can find out with the VarGetType() function. All AutoIt variables are variants. But you can easily convert this double to an integer with the Int() function.

Larsj

Is there any way to look up what each type of variant each propertyidentifier expects? For example, if I want to use the $UIA_NativeWindowHandlePropertyId and pass the windows handle so that I can convert the window handle variable to the appropriate type as i did with the above example per your solution.

Edit : My bad. should have read the microsoft documentation in detail before posting this. I was able to reference the different property identifiers and the expected variant type values for each. Sorry about that.

Edited July 21, 2020 by tsrkumar
Found solution

junkew · July 22, 2020

The most detail you can find in the windows sdk headerfiles.

frank10 · August 24, 2020

First, thanks for UIASpy and the explanations!

Using it to create sample code, often it uses this var: $dtag_IUIAutomationElement

like this for taskbar:

Local $pCondition0
    $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Shell_TrayWnd", $pCondition0 )
    If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
    ConsoleWrite( "$pCondition0 OK" & @CRLF )
    
    Local $pPane1, $oPane1
    $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pPane1 )
    $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
    If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
    ConsoleWrite( "$oPane1 OK" & @CRLF )

But then, with

#include "Includes\CUIAutomation2.au3"

it gives

error: $dtag_IUIAutomationElement: undeclared global variable.

if I put

#include "Includes\UIA_Constants.au3"

it gives other errors on a lot of other vars.

So, I ended up using

#include "Includes\CUIAutomation2.au3"

and modifying the var to

$dtagIUIAutomationElement

and all works fine.

Is it a bug of UIASpy sample code generation?

Or which includes are to be used?

Edited August 24, 2020 by frank10

frank10 · August 24, 2020

I want to click on a program in the TrayNotify, but inspecting it it has no NameProperty nor Classname!

So, how can I directly select it?

I tried this way, of course with no luck, (it goes on the start of the trayNotify on the first Button):

Local $pCondition4
    $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition4 )
    If Not $pCondition4 Then Return ConsoleWrite( "$pCondition4 ERR" & @CRLF )
    ConsoleWrite( "$pCondition4 OK" & @CRLF )
    Local $pButton1, $oButton1
    $oPane3.FindFirst( $TreeScope_Descendants, $pCondition4, $pButton1 )
    $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
    ConsoleWrite( "$oButton1 OK" & @CRLF )
    
    Local $asBoundingRectangle1
    $oButton1.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $asBoundingRectangle1 )
    MouseMove($asBoundingRectangle1[0], $asBoundingRectangle1[1])

$oPane3 was the ToolbarWindow32, correctly selected.

EDIT

I found a way with FindAll and checking for

UIA_NamePropertyId

when it is ""

But this is not so precise... what if one finds 2 buttons without a Name? How could I discriminate them?

Ideally it should be checked against the process Name ID on Task Manager. How?

Edited August 24, 2020 by frank10

PaulChristopher · October 14, 2020

Many thanks to LarsJ and junkew for their work on UI Automation in AutoIt! This is very, very helpful, many thanks!

Alas, I need to automate a poorly accessible application non-unique names, up to 18 sublevels of children etc. To put a long story short: I would like to use the "ElementFromPoint" method the get quickly an element from its screen coordinates.

Does anyone have a working example for the ElementFromPoint method as listed in UIA_Constants.au3?

Here is some example code for Notepad, but it is not working.

$oUIAutomation OK
$oDesktop OK
"\\Client\D$\Sonstiges\AutoIt\sys\_UIAutomation\test_AppNotepad.au3" (34) : ==> The requested action with this object has failed.:
$oDesktop.ElementFromPoint($tStructPoint, $pElement)
$oDesktop^ ERROR

Can anyone give me a hint why the error occurs and how to do it right?

;#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7
;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code
#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Run( "Notepad.exe" )
Sleep( 1000 )
WinSetState( "[CLASS:Notepad]", "", @SW_MAXIMIZE )
Sleep( 1000 )

Example()

Func Example()

   Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
   If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
   ConsoleWrite( "$oUIAutomation OK" & @CRLF )

   ; -- Get desktop element --
   Local $pDesktop, $oDesktop
   $oUIAutomation.GetRootElement( $pDesktop )
   $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
   If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
   ConsoleWrite( "$oDesktop OK" & @CRLF )

   ; --- Find Notepad's main edit field by its screen coordinates ---
   Local $tStructPoint, $pElement, $oElement
   $tStructPoint = DllStructCreate( "int x; int y" )
   DllStructSetData($tStructPoint, "x", 100) ;
   DllStructSetData($tStructPoint, "y", 100)
   $oDesktop.ElementFromPoint($tStructPoint, $pElement)
   ;$oDesktop.ElementFromPoint(DllStructGetPtr($tStructPoint), $pElement)
   If Not $pElement Then Return ConsoleWrite( "$pElement ERR" & @CRLF )
   ConsoleWrite( "$pElement OK" & @CRLF )

   $oElement = ObjCreateInterface( $pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
   If Not IsObj( $oElement ) Then Return ConsoleWrite( "$oElement ERR" & @CRLF )
   ConsoleWrite( "$oElement OK" & @CRLF )

   ; -- Do something with Notepad's main edit control --
   Local $pValuePattern, $oValuePattern
   $oElement.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
   $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtag_IUIAutomationValuePattern )
   If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
   ConsoleWrite( "$oValuePattern OK" & @CRLF )
   ;$oValuePattern.SetValue("Changed!!")
   Local $Value
   $oValuePattern.CurrentValue($Value)
   ConsoleWrite($Value & @CRLF)
EndFunc

InnI · October 14, 2020

41 minutes ago, PaulChristopher said:

how to do it right

;~    $oDesktop.ElementFromPoint($tStructPoint, $pElement)
   $oUIAutomation.ElementFromPoint($tStructPoint, $pElement)

PaulChristopher · October 14, 2020

Thank you so much, InnI! This indeed does the trick! 😊

Decibel · February 13, 2021

Here is a performance tip on finding the top level window from the usual Desktop object. The gain will vary depending on how much stuff is open in your tree (UISpy's left pane).

My assumption on how the tree walker works could be wrong, but this is my observation. The difference is letting the FindAll method drill down through every...single...subelement, or tell it to just go one level down. I wrapped a timer around the two approaches to show the speed gain. Sure, it's milliseconds, but it accumulates the more your script runs through objects.

Low number of windows open:

Timer $TreeScope_Descendants: 124.508
Timer $TreeScope_Children: 9.635

15 extra File Explorer windows open:

Timer $TreeScope_Descendants: 155.358
Timer $TreeScope_Children: 10.055

So again, you have to know for certain that the element that you are looking for is only one level down from the parent, or your element will not be found.

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#AutoIt3Wrapper_UseX64=y

;~ #include "CUIAutomation2.au3"
#include ".\Includes\UIA_Constants.au3" ;<--More recent approach (February 9, 2020 by LarsJ) https://www.autoitscript.com/forum/topic/201683-ui-automation-udfs/

Opt( "MustDeclareVars", 1 )
Opt("WinTitleMatchMode", 2);<--Match any substring in the title (case-sensitive)

Example()

Func Example()
  ; Window handle
  Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" ) ;<-- This grabs the base chrome.exe process, not one with the GUI, so it doesn't work for this example
  If Not IsHWnd( $hWindow ) Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Activate window
  WinActivate( $hWindow )
  Sleep( 100 )

  ; UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation ) ;<--Find & Replace "$dtagIUIAutomation" with "$dtag_IUIAutomation" (underscore difference)
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; Chrome window
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

;~   Local $pChrome, $oChrome
;~   $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pChrome )
;~   $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
;~   If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
;~   ConsoleWrite( "$oChrome OK" & @CRLF )

  Local $pChrome, $oChrome
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pChrome ) ;<-- Using $TreeScope_Children so that it only goes 1 branch deep from $oDesktop
  $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )

  ; --- Read a property to assert the object ---

  Local $vValue
  $oChrome.GetCurrentPropertyValue( $UIA_NamePropertyId, $vValue)
  ConsoleWrite( "$UIA_NamePropertyId: " & $vValue & @CRLF )

EndFunc

TreeScope_Children - The scope includes children of the element.
TreeScope_Descendants - The scope includes children and more distant descendants of the element.

Reference: https://docs.microsoft.com/en-us/windows/win32/api/uiautomationclient/ne-uiautomationclient-treescope

Environment: Windows 10 Home 64-bit 19042.804, Chrome 64-bit Version 88.0.4324.150, AutoIt 3.3.14.5.

Edited February 13, 2021 by Decibel

Decibel · February 13, 2021

When you're working against a client application, not OS objects directly like the System Tray or etc., you can skip the step of getting the Desktop object then using its FindAll method, and go straight to your client application by hWnd which can be retrieved by the good old Windows Management functions like WinGetHandle().

This example comments out the lines that connect to the Desktop object, then use that to connect to the Chrome window. It also shows that you don't need to make Chrome the active window and it will still work fine. We're familiar with that concept from our days of working with Window Management and Controls. By the way, this is also faster than the the other way.

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#AutoIt3Wrapper_UseX64=y

;~ #include "CUIAutomation2.au3"
#include ".\Includes\UIA_Constants.au3" ;<--More recent approach (February 9, 2020 by LarsJ) https://www.autoitscript.com/forum/topic/201683-ui-automation-udfs/

Opt( "MustDeclareVars", 1 )
Opt("WinTitleMatchMode", 2);<--Match any substring in the title (case-sensitive)

Example()

Func Example()
  ; Window handle
;~   Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" ) ;<-- This grabs the base chrome.exe process, not one with the GUI, so it doesn't work for this example
  Local $hWindow = WinGetHandle("[TITLE:Google Chrome; CLASS:Chrome_WidgetWin_1]") ;<-- This can be for any application, not just Chrome that this example is based around
  If Not IsHWnd( $hWindow ) Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

;~   ; Activate window
;~   WinActivate( $hWindow )
;~   Sleep( 100 )

  ; UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation ) ;<--Find & Replace "$dtagIUIAutomation" with "$dtag_IUIAutomation" (underscore difference)
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

;~   ; Desktop element
;~   Local $pDesktop, $oDesktop
;~   $oUIAutomation.GetRootElement( $pDesktop )
;~   $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
;~   If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
;~   ConsoleWrite( "$oDesktop OK" & @CRLF )
;~
;~   ; Chrome window
;~   Local $pCondition
;~   $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition )
;~   If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
;~   ConsoleWrite( "$pCondition OK" & @CRLF )
;~
;~   Local $pChrome, $oChrome
;~   $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pChrome )
;~   $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
;~   If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
;~   ConsoleWrite( "$oChrome OK" & @CRLF )

  ; Get element by handle, without having to first get the UIA Desktop
  Local $oChrome ;<-- repeating this here because it was commented out in the block above
  Local $pElement
  $oUIAutomation.ElementFromHandle($hWindow, $pElement)
  $oChrome = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement9)
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )

  ; --- Read a property to assert the object ---

  Local $vValue
  $oChrome.GetCurrentPropertyValue( $UIA_NamePropertyId, $vValue)
  ConsoleWrite( "$UIA_NamePropertyId: " & $vValue & @CRLF )

EndFunc

Environment: Windows 10 Home 64-bit 19042.804, Chrome 64-bit Version 88.0.4324.150, AutoIt 3.3.14.5.

Edited February 13, 2021 by Decibel

Decibel · February 14, 2021

Doing my part to update us on how the changes in Chrome have caused us to have to change the example script "Chrome - Clicking an extension" which was actually how to open a new tab in Chrome. That answer uses MouseClick because the Invoke method didn't work at that time. I'm going to guess that was a Chrome issue for that older version because it works now. Being able to use the UIA Invoke means that Chrome windows does not have to be the active window, but it will become the active window to the New Tab. This example also uses the approach of setting the client application's UIA object by its hWnd rather than a FindAll from the Desktop object.

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include ".\Includes\UIA_Constants.au3" ;<--More recent approach (February 9, 2020 by LarsJ) https://www.autoitscript.com/forum/topic/201683-ui-automation-udfs/

Opt( "MustDeclareVars", 1 )
Opt("WinTitleMatchMode", 2);Match any substring in the title (case-sensitive)

Example()

Func Example()
    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
    If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
    ConsoleWrite( "$oUIAutomation OK" & @CRLF )

    ;*** Get element by handle, without having to first get the Desktop
    Local $hWindow = WinGetHandle("[TITLE:Google Chrome; CLASS:Chrome_WidgetWin_1]")
    ConsoleWrite( "$hWindow: " & $hWindow & ", " & WinGetTitle($hWindow) & @CRLF )

    Local $pElement, $oChrome
    $oUIAutomation.ElementFromHandle($hWindow, $pElement)
    $oChrome = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement9)
    If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
    ConsoleWrite( "$oChrome OK" & @CRLF )

    ; --- Find New Tab button ---

    ;*** Set up the 2 property condition for the tree search
    Local $pCondition0, $pCondition1, $pAndCondition1
    $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition0 )
    $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "New Tab", $pCondition1 )
    $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
    If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
    ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

    ; Search for and set the object
    Local $pButton1, $oButton1
    $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pButton1 )
    $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement9, $dtag_IUIAutomationElement9 )
    If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
    ConsoleWrite( "$oButton1 OK" & @CRLF )

    ; Invoke pattern will "click" the button without MouseClick
    Local $pInvokePattern1, $oInvokePattern1
    $oButton1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
    $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
    If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
    ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

    $oInvokePattern1.Invoke() ;this is listed in the object's "Control Pattern Methods" list

EndFunc

Environment: Windows 10 Home 64-bit 19042.804, Chrome 64-bit Version 88.0.4324.150, AutoIt 3.3.14.5.

Edited February 14, 2021 by Decibel
Typos

LarsJ · February 14, 2021

A few comments on the 3 posts above.

The first of the 3 posts:
Note that when you generate sample code in UIASpy to find the application top window, this code is created:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pPane1 ) ; FindFirst and $TreeScope_Children
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )
EndFunc

The FindFirst() method is used with the $TreeScope_Children constant.

The second of the 3 posts:
A comparison of FindFirst( $TreeScope_Children, ... ) and WinGetHandle() on my PC with this code:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  Local $hTimer1, $fTime1, $fTimeAll1, $hTimer2, $fTime2, $fTimeAll2

  For $i = 0 To 9
    ConsoleWrite( "$i = " & $i & @CRLF )
  
    $hTimer1 = TimerInit()
    Example1()
    $fTime1 = TimerDiff( $hTimer1 )
    $hTimer1 = 0
    ConsoleWrite( "$fTime1 = " & $fTime1 & @CRLF )
    $fTimeAll1 += $fTime1
    Sleep( 1000 )

    $hTimer2 = TimerInit()
    Example2()
    $fTime2 = TimerDiff( $hTimer2 )
    $hTimer2 = 0
    ConsoleWrite( "$fTime2 = " & $fTime2 & @CRLF & @CRLF )
    $fTimeAll2 += $fTime2
    Sleep( 1000 )
  Next

  ConsoleWrite( "$fTimeAll1 = " & $fTimeAll1 & @CRLF )
  ConsoleWrite( "Average1 = " & $fTimeAll1 / 10 & @CRLF & @CRLF )

  ConsoleWrite( "$fTimeAll2 = " & $fTimeAll2 & @CRLF )
  ConsoleWrite( "Average2 = " & $fTimeAll2 / 10 & @CRLF )
EndFunc

Func Example1()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition0, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )
EndFunc

Func Example2()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" )
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  Local $pElement, $oChrome
  $oUIAutomation.ElementFromHandle( $hWindow, $pElement )
  $oChrome = ObjCreateInterface( $pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )
EndFunc

gives these results:

$fTimeAll1 = 514.211204786465
Average1 = 51.4211204786466

$fTimeAll2 = 200.199417594239
Average2 = 20.0199417594239

The WinGetHandle() code is about 30 milliseconds faster on average.
There may be many good reasons to use the WinGetHandle() function, but I don't think performance considerations are crucial.

Edited February 14, 2021 by LarsJ

Sign In

Using UI Automation Code in AutoIt

Recommended Posts

seadoggie01

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

LarsJ

LarsJ

LarsJ

Posted Images

junkew

seadoggie01

LarsJ

LarsJ

LarsJ

tsrkumar

LarsJ

tsrkumar

tsrkumar

junkew

frank10

frank10

PaulChristopher

InnI

PaulChristopher

Decibel

Decibel

Decibel

LarsJ

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta