Using UI Automation Code in AutoIt

[Earthshine]

Stop trying to make it like standard code. Not going to happen 

[mLipok]

2 hours ago, Earthshine said:

like standard code

I thought that @LarsJ did it as a standard code, it's just a matter of perceiving the standard.

[Earthshine]

oh yeah... he did as a matter of fact. I am different I guess. I like change, not for change's sake, but for the good of my development

[LarsJ]

AtomicSeek, I think it's hard to compare native code and UI Automation code. With regard to the WinExists() function you're certainly getting a lot of functionality in a single line of AutoIt code. That's because all the hard work is implemented in internal C/C++ code. Since the function supports different title matching modes, we're probably talking about hundreds of code lines.

In UI Automation code, there's no internal code to do the hard work. All code must be implemented in pure AutoIt code. That makes a difference.

Regarding the error handling code, I totally agree with BrewManNH that the two ConsoleWrite and also the Return command are important. Especially if you're less experienced in writing UI Automation code.

If you are experienced in writing UI Automation code, you can disable error management code in Sample code | Options | Check error. Then you can add error control only in critical parts of the code through the Sample code | Check error submenu. The code looks like this copied directly from UIASpy code page:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3" ; Get proper version in UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )

    ; Get Desktop element
    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement( $pDesktop )
    $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
EndFunc

; --- Find window/control ---

Local $pCondition0, $pCondition1, $pAndCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Untitled - Notepad", $pCondition1 )
$oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )

Local $pWindow1, $oWindow1
$oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pWindow1 )
$oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

If Not IsObj( $oObject ) Then Return ConsoleWrite( "$oObject ERR" & @CRLF )
ConsoleWrite( "$oObject OK" & @CRLF )

In the #include statements new functionality is introduced which will be published in the weekend.

Note that the code generated in UIASpy is development code as opposed to production code.

In tested and error-free production code, you should definitely delete or comment out all the ConsoleWrite lines. Use only error handling in production code when needed eg. to check that a window exists. And it should be implemented through a MsgBox or similar.

[LarsJ]

Patterns (actions), 13 - 15
Control patterns are objects that can be used to perform actions on an UI element (window/control). The actual actions are performed by executing the methods of the pattern objects.

Control types that are supported by specific control patterns are listed in Control Types and Their Supported Control Patterns.

How to create pattern objects and execute pattern methods through UIASpy is described in How to topics 12 and 14.

The examples shows how to use control patterns. The examples are tested on Windows 10 and Windows 7.

In many examples you need to update eg. control names to your own names. There may also be other values to be updated. Lines to be updated are marked with

; <<<<<<<<<<<<<<<<<<<<

Use UIASpy to check and copy the values.

All code is stored in Examples\4) Patterns (actions)\<Pattern>\ folders.
 

Patterns (actions):

• Section 1 - 12: Examples based on File Explorer
• Section 13 - 15: Virtual listview item patterns
• Section 16 - 17: Other control pattern objects
   

1. File Explorer picture
   
   Window patterns
2. Window Control Pattern
3. Transform Control Pattern
   
   Treeview patterns
4. ExpandCollapse Control Pattern
5. ScrollItem Control Pattern
6. Scroll Control Pattern
   Does not work on Windows 7 ($oScrollPattern1 ERR)
   
   Listview patterns
7. Grid Control Pattern
8. Table Control Pattern
9. Invoke Control Pattern
10. MultipleView Control Pattern
11. SelectionItem Control Pattern
12. Selection Control Pattern
    
    Virtual listview item patterns
13. File Explorer picture
14. ItemContainer Control Pattern  (FindItemByProperty())
15. VirtualizedItem Control Pattern  (Realize())
    
    Other control pattern objects
16. Value Control Pattern
17. RangeValue Control Pattern

 

13. File Explorer picture

 

14. ItemContainer Control pattern
From Microsoft documentation:
The ItemContainer control pattern is used to support item virtualization. The ItemContainer pattern object exposes the FindItemByProperty() method that retrieves an item from a container, such as a virtual list. See top of Microsoft documentation for a detailed description.

Eg. a virtual listview is a container with virtual items.

In a virtual listview only the visible items are realized items represented by listitem automation elements that supports all UI Automation properties. The non-visible items are virtual items represented by placeholder automation elements that only supports a few UI Automation properties.

When virtual items are scrolled into the listview by dragging the scrollbar thumb they are realized (de-virtualized) and becomes visible.

The FindItemByProperty() method of the ItemContainer pattern object is used to retrieve a placeholder automation element for a virtualized item in a virtual container.

The Realize() method of the VirtualizedItem pattern object (next section) takes a placeholder automation element as input and creates a visible (realized, de-virtualized) UI Automation element.

The ItemContainer pattern object is not limited to use by virtualized containers. Any container that can implement efficient name lookup can support this control pattern, enabling clients to look up names more quickly than by using methods such as FindFirst(), which must traverse the Microsoft UI Automation tree.

Preparation
File Explorer should look as shown in the picture. Expand the C-drive in the treeview. Click Windows folder in the treeview to make it visible in the listview. The scrollbar thumb should be positioned at the top of the listview. The SysWOW64 folder should be non-visible. The purpose of this example and the next example is to realize (de-virtualize) the SysWOW64 folder to make it visible in the listview. Open UIASpy, place the mouse over the topmost item in the listview, press F2, press F7. The listview should be detected. F2 detects the listview item that's parent for the individual item cells (image and edit controls). F7 navigates to the parent control of the item which is the listview.

Code
The UIASpy created code shows how to get the SysWOW64 folder in the listview as a placeholder automation element with the FindItemByProperty() method of the ItemContainer pattern object (ItemContainer.au3, ItemContainer-a.au3 contains code directly from UIASpy):

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\CUIAutomation2.au3" ; Get proper version in UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0, $pCondition1, $pAndCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CabinetWClass", $pCondition0 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Windows", $pCondition1 ) ; <<<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
  If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  Local $pList1, $oList1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition2, $pList1 )
  $oList1 = ObjCreateInterface( $pList1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oList1 ) Then Return ConsoleWrite( "$oList1 ERR" & @CRLF )
  ConsoleWrite( "$oList1 OK" & @CRLF )

  ; --- ItemContainer Pattern (action) Object ---

  ConsoleWrite( "--- ItemContainer Pattern (action) Object ---" & @CRLF )

  Local $pItemContainerPattern1, $oItemContainerPattern1
  $oList1.GetCurrentPattern( $UIA_ItemContainerPatternId, $pItemContainerPattern1 )
  $oItemContainerPattern1 = ObjCreateInterface( $pItemContainerPattern1, $sIID_IUIAutomationItemContainerPattern, $dtagIUIAutomationItemContainerPattern )
  If Not IsObj( $oItemContainerPattern1 ) Then Return ConsoleWrite( "$oItemContainerPattern1 ERR" & @CRLF )
  ConsoleWrite( "$oItemContainerPattern1 OK" & @CRLF )

  ; --- ItemContainer Pattern (action) Methods ---

  ConsoleWrite( "--- ItemContainer Pattern (action) Methods ---" & @CRLF )

  Local $pElement
  $oItemContainerPattern1.FindItemByProperty( NULL, $UIA_NamePropertyId, "SysWOW64", $pElement ) ; <<<<<<<<<
  ConsoleWrite( "$oItemContainerPattern1.FindItemByProperty()" & @CRLF )
  If Not $pElement Then Return ConsoleWrite( "$pElement ERR" & @CRLF )
  ConsoleWrite( "$pElement OK" & @CRLF )

  ; --- Copy element info ---

  ; $UIA_NamePropertyId                                 SysWOW64
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pAndCondition1 OK
$oWindow1 OK
--- Find window/control ---
$pCondition2 OK
$oList1 OK
--- ItemContainer Pattern (action) Object ---
$oItemContainerPattern1 OK
--- ItemContainer Pattern (action) Methods ---
$oItemContainerPattern1.FindItemByProperty()
$pElement OK

Remarks
Read about $oItemContainerPattern.FindItemByProperty() in the Microsoft documentation.
 

15. VirtualizedItem Control pattern
From Microsoft documentation:
The VirtualizedItem control pattern is used to support virtualized items, which are items that are represented by placeholder automation elements in the Microsoft UI Automation tree.

Eg. a virtual (non-visible) item in a virtual listview.

See the extract of the Microsoft documentation in the section above about the ItemContainer control pattern.

Preparation
The example is a continuation of the example in the section above. Prepare and run the example above and make sure that the placeholder automation element ($pElement in bottom of script) is created without errors. In this example the virtual and non-visible SysWOW64 folder ($pElement) is realized (de-virtualized) to make it visible at the bottom of the listview.

Code
The code is a continuation of the code in the section above about the ItemContainer pattern object and shows how to use the Realize() method of the VirtualizedItem pattern object to make the SysWOW64 folder visible in the listview. Add code with UIASpy to create the VirtualizedItem pattern object and to execute the Realize() method (VirtualizedItem.au3, VirtualizedItem-a.au3 contains code directly from UIASpy):

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\CUIAutomation2.au3" ; Get proper version in UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0, $pCondition1, $pAndCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CabinetWClass", $pCondition0 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Windows", $pCondition1 ) ; <<<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
  If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  Local $pList1, $oList1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pCondition2, $pList1 )
  $oList1 = ObjCreateInterface( $pList1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oList1 ) Then Return ConsoleWrite( "$oList1 ERR" & @CRLF )
  ConsoleWrite( "$oList1 OK" & @CRLF )

  ; --- ItemContainer Pattern (action) Object ---

  ConsoleWrite( "--- ItemContainer Pattern (action) Object ---" & @CRLF )

  Local $pItemContainerPattern1, $oItemContainerPattern1
  $oList1.GetCurrentPattern( $UIA_ItemContainerPatternId, $pItemContainerPattern1 )
  $oItemContainerPattern1 = ObjCreateInterface( $pItemContainerPattern1, $sIID_IUIAutomationItemContainerPattern, $dtagIUIAutomationItemContainerPattern )
  If Not IsObj( $oItemContainerPattern1 ) Then Return ConsoleWrite( "$oItemContainerPattern1 ERR" & @CRLF )
  ConsoleWrite( "$oItemContainerPattern1 OK" & @CRLF )

  ; --- ItemContainer Pattern (action) Methods ---

  ConsoleWrite( "--- ItemContainer Pattern (action) Methods ---" & @CRLF )

  Local $pElement, $oElement
  $oItemContainerPattern1.FindItemByProperty( NULL, $UIA_NamePropertyId, "SysWOW64", $pElement ) ; <<<<<<<<<
  ConsoleWrite( "$oItemContainerPattern1.FindItemByProperty()" & @CRLF )
  If Not $pElement Then Return ConsoleWrite( "$pElement ERR" & @CRLF )
  ConsoleWrite( "$pElement OK" & @CRLF )
  $oElement = ObjCreateInterface( $pElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  ; --- Copy element info ---

  ; $UIA_NamePropertyId                                 SysWOW64

  ; --- VirtualizedItem Pattern (action) Object ---

  ConsoleWrite( "--- VirtualizedItem Pattern (action) Object ---" & @CRLF )

  Local $pVirtualizedItemPattern1, $oVirtualizedItemPattern1
  $oElement.GetCurrentPattern( $UIA_VirtualizedItemPatternId, $pVirtualizedItemPattern1 )
  $oVirtualizedItemPattern1 = ObjCreateInterface( $pVirtualizedItemPattern1, $sIID_IUIAutomationVirtualizedItemPattern, $dtagIUIAutomationVirtualizedItemPattern )
  If Not IsObj( $oVirtualizedItemPattern1 ) Then Return ConsoleWrite( "$oVirtualizedItemPattern1 ERR" & @CRLF )
  ConsoleWrite( "$oVirtualizedItemPattern1 OK" & @CRLF )

  ; --- VirtualizedItem Pattern (action) Methods ---

  ConsoleWrite( "--- VirtualizedItem Pattern (action) Methods ---" & @CRLF )

  $oVirtualizedItemPattern1.Realize()
  ConsoleWrite( "$oVirtualizedItemPattern1.Realize()" & @CRLF )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pAndCondition1 OK
$oWindow1 OK
--- Find window/control ---
$pCondition2 OK
$oList1 OK
--- ItemContainer Pattern (action) Object ---
$oItemContainerPattern1 OK
--- ItemContainer Pattern (action) Methods ---
$oItemContainerPattern1.FindItemByProperty()
$pElement OK
--- VirtualizedItem Pattern (action) Object ---
$oVirtualizedItemPattern1 OK
--- VirtualizedItem Pattern (action) Methods ---
$oVirtualizedItemPattern1.Realize()

Remarks
Note the corrections (compare VirtualizedItem.au3 and VirtualizedItem-a.au3) in the last part of the code (from the "ItemContainer Pattern (action) Methods" section). With UIASpy tool you can only spy on elements that are actually visible. Therefore, it isn't possible to obtain accurate information on non-visible virtual elements.

Edited February 9, 2020 by LarsJ
Text updates

[LarsJ]

Code snippets
The purpose of these examples is to show how to use Code snippets in UIASpy.

1. Create an UI Automation element array from pointer  (MenuBar, CreateTrueCondition())
2. Activate (give focus to) a window: How to topics number 16  (MenuItem)
3. Mouse click in middle of bounding rectangle: How to topics number 16  (MenuItem)
4. Control click via PostMessage and $UIA_AutomationIdPropertyId: How to topics number 17  (MenuItem)

All code is stored in Examples\5) Code snippets\ folder.

 

1. Create an UI Automation element array from pointer
Preparation
Open an empty Notepad. Open UIASpy, place the mouse over empty space in the menu bar, press F1.

Code
The code created with UIASpy shows how to use FindAll() to get a pointer to an element array of menu items, how to create the element array object and how to traverse the array (ElementArray.au3, ElementArray-a.au3 contains code directly from UIASpy):

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\CUIAutomation2.au3" ; Get proper version in UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pWindow1, $oWindow1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow1 )
  $oWindow1 = ObjCreateInterface( $pWindow1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow1 ) Then Return ConsoleWrite( "$oWindow1 ERR" & @CRLF )
  ConsoleWrite( "$oWindow1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1, $pCondition2, $pAndCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuBarControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Application", $pCondition2 ) ; <<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 )
  If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition2 OK" & @CRLF )

  Local $pMenuBar1, $oMenuBar1
  $oWindow1.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pMenuBar1 )
  $oMenuBar1 = ObjCreateInterface( $pMenuBar1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuBar1 ) Then Return ConsoleWrite( "$oMenuBar1 ERR" & @CRLF )
  ConsoleWrite( "$oMenuBar1 OK" & @CRLF )

  ; --- $oUIAutomation methods ---

  ConsoleWrite( "--- $oUIAutomation methods ---" & @CRLF )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  ConsoleWrite( "$oUIAutomation.CreateTrueCondition()" & @CRLF )

  ; --- $oUIElement methods ---

  ConsoleWrite( "--- $oUIElement methods ---" & @CRLF )

  Local $pElements
  $oMenuBar1.FindAll( $TreeScope_Descendants, $pCondition, $pElements )
  ConsoleWrite( "$oMenuBar1.FindAll()" & @CRLF )

  ; --- Code Snippets ---

  ConsoleWrite( "--- Code Snippets ---" & @CRLF )

  Local $oUIElementArray1, $iLength1 ; $pElements is a pointer to an UI Automation element array
  $oUIElementArray1 = ObjCreateInterFace( $pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray1.Length( $iLength1 )
  If Not $iLength1 Then Return ConsoleWrite( "$iLength1 = 0 ERR" & @CRLF )
  ConsoleWrite( "$iLength1 = " & $iLength1 & @CRLF )

  ; --- Code Snippets ---

  ConsoleWrite( "--- Code Snippets ---" & @CRLF )

  Local $pElement1, $oElement1, $sValue1
  For $i = 0 To $iLength1 - 1
    $oUIElementArray1.GetElement( $i, $pElement1 )
    $oElement1 = ObjCreateInterface( $pElement1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    ;$oElement1.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sValue1 ) ; $UIA_ClassNamePropertyId is used as example
    ;ConsoleWrite( "$sValue1 = " & $sValue1 & @CRLF )
    $oElement1.GetCurrentPropertyValue( $UIA_NamePropertyId, $sValue1 )
    ConsoleWrite( "$sValue1 = " & $sValue1 & @CRLF )
  Next

  ; --- Copy element info ---

  ; $UIA_NamePropertyId                                 File
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oWindow1 OK
--- Find window/control ---
$pAndCondition2 OK
$oMenuBar1 OK
--- $oUIAutomation methods ---
$oUIAutomation.CreateTrueCondition()
--- $oUIElement methods ---
$oMenuBar1.FindAll()
--- Code Snippets ---
$iLength1 = 5
--- Code Snippets ---
$sValue1 = File
$sValue1 = Edit
$sValue1 = Format
$sValue1 = View
$sValue1 = Help

Remarks
Note that CreateTrueCondition() is created with UIASpy | Sample code | Objects/methods | Automation object methods... and that FindAll() is created with UIASpy | Sample code | Objects/methods | Element object methods...

Note also the loop in bottom of the code that's used to traverse the UI Automation element array to get information about the individual elements. $UIA_NamePropertyId is copied from the first menu item in UIASpy | Sample code | Properties...

Examples
Other examples that demonstrates the use of UI Automation element arrays:

• The Table Control Pattern example: Patterns (actions) section 8
• The Selection Control Pattern example: Patterns (actions) section 12
• Get all texts in a Chrome window

Edited April 25, 2019 by LarsJ

[Mbee]

Most Respected @LarsJ, I am approximately infinitely impressed with the staggeringly great work you've done here!  Wow!

I recently (re-) started a thread, here: Updating question about manipulating non-standard GUIs. Your amazing system was recommended to help me reach my goal of automating Comodo Free Firewall.  I was excited by the power you've provided and eagerly began right at the start of your tutorials...

In your OP you say: "Is this approach difficult? Not at all as the examples will show."  Regrettably however, I'm apparently exceptionally stupid, because I'm rather lost!

I read just what you recommended, starting with the MSDN / Microsoft Docs, and then proceeded with several of your first examples in the tutorial. Here's where I am...

The concepts are crystal clear, but I found that I could only follow the examples blindly -- I never had a firm, clear understanding of the very core of the code generation: The WHY? Why choose this aspect or area of the UIASpy output instead of that? Why does the generated code seem so arbitrary? I could generate code from pretty much anywhere (there's a HELL of a lot of redundancy) -- why did you choose specifically what you did?

More to the point right now for me: How do you determine which state a particular region of the non-standard GUI is in -- that is, read the state as if it were a control (which only seems to be true within UIASpy)?

Before long, I was so deeply puzzled and confused by your system that I had a mini mental breakdown and went back to trying to do things with the extremely crude -- but functional -- technique of MouseClick() calls.

Perhaps you could show me what steps I should take for just that one step of manipulating Comodo Free firewall?   I'd provide my code (in fact, I already have done so for an earlier version), but I have never been able to generate anything meaningful at all with UIASpy.  Kindly take pity on a moronic newb and advise me please?

Thanks!

[Mbee]

You know, perhaps I'd be more comfortable and fare better if I began coding the object instantiation/initialization and method calls manually rather than having UIASpy generate it for me.  I would use UIASpy to identify the parameter values and the like, but at least at first,the bulk of the coding would be done manually. It's obviously possible, but what matters most is whether this is practical for a newb like myself. I didn't see a UDF to download in your thread, so would I use @junkew's stuff?

What is your far more informed opinion, sir?

Thank you

[junkew]

Automation is just

1. Identify your object of interest

2. Do an action on it

Difficulty only starts when you need more flexibility in recognition. Uia from microsoft is nicely documented but you need some dev skills to deal with this. Larsj made in this thead excellent examples on the raw coding. My uiawrappers as it says wrapped away some complexity. Comparing to selenium or mobile automation it all comes back to above 2 points and a decent spy tool to help you out.

[LarsJ]

Mbee, I think that modern GUIs is a common term for what you call non-standard GUIs.

In order to be able to use UI Automation code, it's necessary at least to have an idea of what it's all about. There are links to Microsoft documentation in the help system in UIASpy.

You can also read the How to topics which is a short introduction to both UI Automation code and the UIASpy tool. There are three posts in total. The first is a general introduction to UI Automation code. The second is about using the UIASpy tool. You can skip the third post. Add a new post if you have any questions. But please be specific.

It's true that there is some redundancy regarding both UIASpy and UI Automation data. In UIASpy you can do the same things in several different ways. And you can also get the same UI Automation data in several different ways.

It's also correct that I have not made a UDF in the same way as junkews UIAWrappers.au3.

The idea of my project is precisely to avoid such a UDF. Instead, I want to use the Microsoft objects and methods directly only translated into AutoIt code.

In UIASpy you can see what information is available to identify and find a specific window/control and you can see which patterns (actions) you can use. You can generate code to see eg. how the ObjCreateInterface command should look.

Some of the more advanced object methods use safearrays as either input or output parameters. UIASpy can generate code to handle these safearrays.

For very simple examples like my demo examples you can generate pretty accurate code with UIASpy. For more advanced automation tasks such as automating Comodo, it's necessary to put the different code snippets together into a final script.

Why do I want to avoid a UDF like junkews UIAWrappers.au3? The whole idea of my project has been to add the new UI Automation features that have been added through Windows 8, 8.1 and 10.

UIAWrappers.au3 contains functions. CUIAutomation2.au3 contains Windows 7 constants and definitions. All code can be used from Windows XP to Windows 10. But only Windows 7 constants, definitions and objects are supported. There are 2 main objects: $oUIAutomation and $oUIAutomationElement.

The new features in Windows 8 - 10 are implemented through several new main objects that depends on a particular Windows version. It includes $oUIAutomation2, $oUIAutomation3, $oUIAutomation4, $oUIAutomation5, $oUIAutomationElement2, $oUIAutomationElement3, $oUIAutomationElement4, $oUIAutomationElement5, $oUIAutomationElement6, $oUIAutomationElement7 and $oUIAutomationElement8.

If you have looked at the code in UIAWrappers.au3 (about 1,000 real code lines) then it's not difficult to imagine that integrating all these new objects into the code and taking into account different Windows versions is not quite trivial. And that's very mildly said. It'll be a huge task that's close to impossible.

In my project, it's much easier. I can just add the new objects and methods to the existing arrays, and add code snippets to create the objects and methods. But I'll leave it to the user of UIASpy to select the objects that fit the Windows version and put all the code pieces together into a final script.

I'll take a closer look at Comodo, but I'll probably need a week or two.

[Mbee]

Thank you most sincerely , @LarsJ!  I am extremely grateful for your clearly very valuable time. Honored, even.

I'm not at my best today, but I assure you I will re-dedicate myself to learning from your wonderfully long reply and studying the situation more extensively.  PLEASE don't hurry on my account -- I feel guilty enough for even having asked...

Many thanks!

[LarsJ]

Real examples
These are real examples as opposed to the demo examples. A single example is reviewed below. Otherwise, there's just a link to the example in the General Help and Support forum.

1. Chrome - Activate a tab
2. Get all texts in a Chrome document
3. Automating Comodo Firewall menus
4. Changing text on a control that only shows up when you click in it
5. Detect a 'button' that only appears as a progress bar object

All code is stored in Examples\6) Real examples\ folder.
 

3. Automating Comodo Firewall menus
This is an automation of the menu system in the Comodo Firewall as described in this thread.

The menu system that you can see on the left side of the images is a modern-looking menu system. But each menu item is implemented as a set of standard controls: An icon on the left, a text field in the middle and a button.

The button is placed slightly differently from one menu item to another. In the second picture below, the button is located to the far right.

Comodo probably uses this implementation with standard controls to ensure that the same GUI can be used on all Windows versions from Windows XP to Windows 10.

The menu system can most likely be automated with classic automation code. But you need a UI spy tool that is able to reveal this structure of the menu items.

Because it's about a Firewall, all AutoIt code is immediately blocked. As you can see in the first picture below, all firewall features are therefore disabled, so it's possible to run AutoIt code.

Automation of the menu system with UI Automation code and UIASpy is straightforward.

The task is initially divided into 3 small parts and 3 small scripts:

1. Click Settings button to open Advanced Settings
2. Click Firewall menu item to open the submenu
3. Click Application Rules to access these settings

This procedure is described in How to topics number 1.

As the last task the 3 scripts are put together into a final script.

The 3 small sub-tasks are implemented in completely the same way:

1. Create UI Automation main objects (How to topics number 8)
2. Identify and find window (How to topics number 9)
3. Identify and find button (How to topics number 10)
4. Create the Invoke object (How to topics number 12)
5. Perform the Invoke() method (How to topics number 14)
6. Create executable code (How to topics number 15)

The easiest is to do it all using UIASpy. Operation of UIASpy is described directly in the UIASpy help system.

Note that automation is done on a Windows 10 PC, but that Windows mode in UIASpy is set to Windows 7 mode (Options | Windows | Mode) to be able to generate Windows 7 code.

To examine the structure of the controls in the menu items (necessary to identify the button in point 3), it's easiest to generate the tree structure for the whole window and paste the structure into a new page in your editor so that you can see the structure while using UIASpy.

To generate executable code in point 6, it's only necessary to add the path to UIA_Constants.au3 and to move the different code snippets into the function.

You can find the code in Examples\6) Real examples\ folder. The a-versions of the files contains the code generated directly by UIASpy.

 

1. Click Settings

To run the code, the Comodo GUI should look as shown in the picture. Comodo1.au3:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CisMainWizard", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "SettingsButton", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pButton1, $oButton1
  $oPane1.FindFirst( $TreeScope_Descendants, $pCondition1, $pButton1 )
  $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
  ConsoleWrite( "$oButton1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern1, $oInvokePattern1
  $oButton1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
EndFunc

Output in SciTE console:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oPane1 OK
--- Find window/control ---
$pCondition1 OK
$oButton1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()

 

2. Click Firewall menu item

To run the code, the Comodo GUI should look as shown in the picture. Comodo2.au3:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0, $pCondition1, $pAndCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "COMODO Advanced Settings", $pCondition0 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CisMainWizard", $pCondition1 )
  $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
  If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition2, $pCondition3, $pAndCondition3
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "btnAseSlide", $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition3 )
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  Local $pCondition4, $pAndCondition4
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Firewall Button", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pAndCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  Local $pButton1, $oButton1
  $oPane1.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pButton1 )
  $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
  ConsoleWrite( "$oButton1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern1, $oInvokePattern1
  $oButton1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
EndFunc

Output in SciTE console:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pAndCondition1 OK
$oPane1 OK
--- Find window/control ---
$pAndCondition3 OK
$pAndCondition4 OK
$oButton1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()

 

3. Click Application Rules

To run the code, the Comodo GUI should look as shown in the picture. Comodo3.au3:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0, $pCondition1, $pAndCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "COMODO Advanced Settings", $pCondition0 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CisMainWizard", $pCondition1 )
  $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
  If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition2, $pCondition3, $pAndCondition3
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "A6", $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition3 )
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  Local $pCondition4, $pAndCondition4
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Application Rules Link", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pAndCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  Local $pHyperlink1, $oHyperlink1
  $oPane1.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pHyperlink1 )
  $oHyperlink1 = ObjCreateInterface( $pHyperlink1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oHyperlink1 ) Then Return ConsoleWrite( "$oHyperlink1 ERR" & @CRLF )
  ConsoleWrite( "$oHyperlink1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern1, $oInvokePattern1
  $oHyperlink1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
EndFunc

The Comodo GUI should now look as shown in the picture:

Output in SciTE console:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pAndCondition1 OK
$oPane1 OK
--- Find window/control ---
$pAndCondition3 OK
$pAndCondition4 OK
$oHyperlink1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()

 

All code
To put all the code together in a single script start by copying Comodo1.au3 to ComodoAll.au3. Add a Sleep( 1000 ) and add the code from Comodo2.au3 (except function header and Automation main objects). Sleep( 1000 ) is required for Comodo to have time to open the child window. The 1000 milliseconds can probably be optimized for something less.

Test the code. In SciTE console you'll see warnings about double-declared local variables. Delete these declarations.

Add a Sleep( 1000 ). Add the code from Comodo3.au3. Because the window in step 2 and step 3 is the same, you don't need to identify and find the window again.

Test the code. In SciTE console you'll see warnings about double-declared local variables. Delete these declarations.

To run all code, the Comodo GUI should look as shown in the first picture above. ComodoAll.au3:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CisMainWizard", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pPane1, $oPane1
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "SettingsButton", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pButton1, $oButton1
  $oPane1.FindFirst( $TreeScope_Descendants, $pCondition1, $pButton1 )
  $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
  ConsoleWrite( "$oButton1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern1, $oInvokePattern1
  $oButton1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
  Sleep( 1000 )




  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pAndCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "COMODO Advanced Settings", $pCondition0 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "CisMainWizard", $pCondition1 )
  $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
  If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

  $oDesktop.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pPane1 )
  $oPane1 = ObjCreateInterface( $pPane1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oPane1 ) Then Return ConsoleWrite( "$oPane1 ERR" & @CRLF )
  ConsoleWrite( "$oPane1 OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition2, $pCondition3, $pAndCondition3
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "btnAseSlide", $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition3 )
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  Local $pCondition4, $pAndCondition4
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Firewall Button", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pAndCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  $oPane1.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pButton1 )
  $oButton1 = ObjCreateInterface( $pButton1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oButton1 ) Then Return ConsoleWrite( "$oButton1 ERR" & @CRLF )
  ConsoleWrite( "$oButton1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  $oButton1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
  Sleep( 1000 )




  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "A6", $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition3 )
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Application Rules Link", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pAndCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  Local $pHyperlink1, $oHyperlink1
  $oPane1.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pHyperlink1 )
  $oHyperlink1 = ObjCreateInterface( $pHyperlink1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oHyperlink1 ) Then Return ConsoleWrite( "$oHyperlink1 ERR" & @CRLF )
  ConsoleWrite( "$oHyperlink1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  $oHyperlink1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
  $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern1.Invoke()
  ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )
EndFunc

Output in SciTE console:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oPane1 OK
--- Find window/control ---
$pCondition1 OK
$oButton1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()
--- Find window/control ---
$pAndCondition1 OK
$oPane1 OK
--- Find window/control ---
$pAndCondition3 OK
$pAndCondition4 OK
$oButton1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()
--- Find window/control ---
$pAndCondition3 OK
$pAndCondition4 OK
$oHyperlink1 OK
--- Invoke Pattern (action) Object ---
$oInvokePattern1 OK
--- Invoke Pattern (action) Methods ---
$oInvokePattern1.Invoke()

 

In order to create the code for such an automation task, you don't have to write a single line of automation code yourself. UIASpy generates all code. You don't have to do anything else than clicking around in UIASpy, and do a few copy/paste operations. Of course, you must be able to identify all windows and controls, but this is not different from classic automation.

So far, I'll maintain the statement from first post: Is this approach (using Microsoft objects and methods directly without a set of intermediate functions in a UDF) difficult? Not at all as the examples show.

Edited April 27, 2019 by LarsJ
Item 2 in top of post

[GHOSTSKIKDA]

How Spy FM20.dll Components in VB6 ???

[Earthshine]

LVb6 Apps usually does not require UIA. Standard autoit should work

Edited April 6, 2019 by Earthshine

[junkew]

Start with a spy app what can be recognized highlighted.

[GHOSTSKIKDA]

Try it yourself, the same problem does not appear

GENERATOR.exe

[argumentum]

4 hours ago, GHOSTSKIKDA said:

Try it yourself,

[LarsJ]

GHOSTSKIKDA, It's incredibly so interested you are in automating exactly this control. What is your legal purpose for this? You just want to help people who have unfortunately forgotten what they have entered?

[GHOSTSKIKDA]

 

3 hours ago, LarsJ said:

GHOSTSKIKDA, It's incredibly so interested you are in automating exactly this control. What is your legal purpose for this? You just want to help people who have unfortunately forgotten what they have entered? 

I want to help people solve this problem because they are using older programs

 

Edited April 7, 2019 by GHOSTSKIKDA

[GHOSTSKIKDA]

13 hours ago, argumentum said:

You are missing a file " FM20.dll "   .