IUIAutomation MS framework automate chrome, FF, IE, ....

[junkew]

@larsj made Some examples on virtualized lists and how to handle them

[rcmaehl]

Having some issue with this UDF working correctly on multiple monitors.

[LarsJ]

Windows Start menu

Over the past year there have been a number of questions related to Windows 10 Start menu, for example this.

The code below prints the controls in the Start menu in a hierarchical structure like a treeview. Output in SciTE console. The code is tested on Windows 10, 7 and XP. On Windows 10 I've tested version 1511 and 1607. There are some differences in the Start menu between these versions.

(Windows 10 was released in july 2015. So far there have been two major updates. An update in november 2015 and an update in juli 2016 (the anniversary update). In Windows 10 major updates are provided with a version number. The november 2015 update is version 1511 (yymm) and the juli 2016 update is version 1607. The first release is version 1507. You can see the version number in Start menu | Settings | Update & security when updates are downloaded and installed.)

On Windows 10 the class name of the Start menu is "Windows.UI.Core.CoreWindow". The class name of the Search menu is also "Windows.UI.Core.CoreWindow". The code works for both the Start menu and the Search Menu. (In a standard configuration the Search button is located in the taskbar just to the right of the Start button.)

On Windows 7 and XP the class name of the Start menu is "DV2ControlHost".

Output in SciTE console depends on which windows and menus you have activated in the Start menu (or Search menu on Windows 10). You can have very different output. And generally you will see a lot of output.

expandcollapse popup

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

ConsoleWrite( "5 seconds to open Start menu..." & @CRLF )
Sleep( 5000 )

Example()

Func Example()
  ; Vindows version
  Local $sOS = @OSVersion, $f10 = ( $sOS = "WIN_10" ), $f7 = ( $sOS = "WIN_7" ), $fXP = ( $sOS = "WIN_XP" ), $fOS = $f10?1:$f7?1:$fXP?1:0
  If Not $fOS Then Return ConsoleWrite( "@OSVersion ERR" & @CRLF )
  ConsoleWrite( "@OSVersion OK" & @CRLF )

  ; Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop object
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; Find condition
  Local $pCondition ; Use Inspect.exe (Inspect Objects) or Simple spy demo to find the class name
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, $f10?"Windows.UI.Core.CoreWindow":"DV2ControlHost", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find Start menu
  Local $pStartMenu, $oStartMenu
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pStartMenu )
  $oStartMenu = ObjCreateInterface( $pStartMenu, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oStartMenu ) Then Return ConsoleWrite( "$oStartMenu ERR" & @CRLF )
  ConsoleWrite( "$oStartMenu OK" & @CRLF )

  ; Print info
  ConsoleWrite( @CRLF )
  ConsoleWrite( "Title     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_NamePropertyId ) & @CRLF & _
                "Class     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_ClassNamePropertyId ) & @CRLF & _
                "Ctrl type = " & GetCurrentPropertyValue( $oStartMenu, $UIA_ControlTypePropertyId ) & @CRLF & _
                "Ctrl name = " & GetCurrentPropertyValue( $oStartMenu, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                "Value     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                "Handle    = " & Hex( GetCurrentPropertyValue( $oStartMenu, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

  ; List all child elements
  ListDescendants( $oStartMenu, 0, 0, "    " )

  ; On Windows 7 and XP popup menus are not accessible directly from the Start menu. They
  ; are only accessible from the desktop. Therefore, it's necessary with a separate search.

  Switch True
    Case $f10
      Return
    Case $f7, $fXP
      ; Find condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find Popup menu
      Local $pPopupMenu, $oPopupMenu
      $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pPopupMenu )
      $oPopupMenu = ObjCreateInterface( $pPopupMenu, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oPopupMenu ) Then Return ConsoleWrite( "$oPopupMenu ERR" & @CRLF )
      ConsoleWrite( "$oPopupMenu OK" & @CRLF )

      ; Print info
      ConsoleWrite( @CRLF )
      ConsoleWrite( "Title     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_NamePropertyId ) & @CRLF & _
                    "Class     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_ClassNamePropertyId ) & @CRLF & _
                    "Ctrl type = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_ControlTypePropertyId ) & @CRLF & _
                    "Ctrl name = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                    "Value     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                    "Handle    = " & Hex( GetCurrentPropertyValue( $oPopupMenu, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

      ; List all child elements
      ListDescendants( $oPopupMenu, 0, 0, "    " )
  EndSwitch
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0, $sIndent = "" )
  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd
EndFunc

Func GetCurrentPropertyValue( $oObject, $iPropertyId )
  Local $vValue, $sString
  $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue )
  If Not IsArray( $vValue ) Then Return $vValue
  $sString = $vValue[0]
  For $i = 1 To UBound( $vValue ) - 1
    $sString &= "; " & $vValue[$i]
  Next
  Return $sString
EndFunc

 

I'm testing the Windows version (10, 7, XP) with code like this:

Local $sOS = @OSVersion, $f10 = ( $sOS = "WIN_10" )

If you are using Windows 10 with an old version of AutoIt eg. 3.3.10 the "WIN_10" string is not registered in the @OSVersion macro and the Windows 10 version test will not work. If this is the case just replace

$f10 = ( $sOS = "WIN_10" )

with

$f10 = 1

 

Edited October 8, 2016 by LarsJ
IUIAutomation::GetRootElement

[junkew]

@Larsj any specific reason you use 

  Local $hDesktop = _WinAPI_GetDesktopWindow()

instead of 

IUIAutomation::GetRootElement method

[LarsJ]

I had forgotten that method. Code corrected.

[Danyfirex]

Hello. @LarsJ it's Working right over W10. I love this part ($fOS = $f10?1:$f7?1:$fXP?1:0)  lol

 

Saludos

[HighlanderSword]

Hello,

 

I go the process to click on the file button in outlook, now trying to figure out how to click on account settings drop down box , and

then select account settings from the dropdown box

 

I have tried the below and have had no luck, I'm trying to learn this process so i can use it

 

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

Example()

Func Example()
  Local $hWindow = WinGetHandle( "Mail to Read" )
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF )

  ; Condition to find split button
  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Condition to find "Lists"
  ; "Lists" is a localized word
  ; Use the word in your own language
  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Account Settings", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  ; Both conditions
  Local $pCondition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find "Lists" split button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; Click (Invoke) button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

and in the console i get

 

$hWindow OK
$oUIAutomation OK
$oWindow OK
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oButton ERR

 

 

 

Edited November 8, 2016 by HighlanderSword

[LarsJ]

Are you sure that this dropdown box is a button and not a combo box? Have you checked with the spy tool or Inspect.exe?

You can also print information with code like this:

expandcollapse popup

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

Example()


; List all automation elements of a window
; in a hierarchical structure like a treeview.
Func Example()
  ; Get window handle
  ;Local $hWindow = 0x0006045C                                                             ; Infragistics
  ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Infragistics
  ;Local $hWindow = WinGetHandle( "[REGEXPCLASS:WindowsForms10.Window.8.app.*]" )          ; Windows Updates Downloader
  ;Local $hWindow = WinGetHandle( "BCGPVisualStudioGUIDemo - Start Page" )                 ; BCGSoft BCGPVisualStudioGUIDemo
  ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )                                ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )                                ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )                          ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" )         ; AutoIt GUI window
  ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" )                                ; AutoIt script
  ;Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" )                           ; Chrome
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                                      ; Internet Explorer
  Local $hWindow = WinGetHandle( "[CLASS:WordPadClass]" )                                 ; WordPad
  ;Local $hWindow = WinGetHandle( "Calculator" )                                           ; Calculator
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF & @CRLF )

  ; List all elements of window
  ListDescendants( $oWindow, 0 )
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0, $sIndent = "" )
  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd
EndFunc

Func GetCurrentPropertyValue( $oObject, $iPropertyId )
  Local $vValue, $sString
  $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue )
  If Not IsArray( $vValue ) Then Return $vValue
  $sString = $vValue[0]
  For $i = 1 To UBound( $vValue ) - 1
    $sString &= "; " & $vValue[$i]
  Next
  Return $sString
EndFunc

Use the proper window. You can add a few seconds Sleep as first line in Example function if you need time to open a menu or child window.

Edited November 9, 2016 by LarsJ

[HighlanderSword]

Hello,

 

Below is what I got from Inspector.

 

And Simplespy

 

Regards

Paul

 

Inspector :

How found:    Focus [o:0x60200AA,c:0x0]
    hwnd=0x00000000000206D0 64bit class="NetUIHWND" style=0x56030000 ex=0x0
ChildId:    0
Interfaces:    
Impl:    Remote native IAccessible
Name:    "Account Settings..."
Value:    [null]
Role:    drop down button (0x38)
State:    focused,pressed,focusable,has popup (0x4010000C)
Location:    {l:180, t:207, w:97, h:86}
Selection:    [Error: calling getter for this property: hr=0xFFFFFFFF80020003 - Member not found.]
Description:    [null]
Kbshortcut:    "Alt, F, I, S"
DefAction:    "Open"
Help:    [null]
HelpTopic:    ""
ChildCount:    0
Window:    0x206D0
FirstChild:    [null]
LastChild:    [null]
Next:    "Account Settings" : text : read only
Previous:    [null]
Left:    [null]
Up:    [null]
Right:    "Account Settings" : text : read only
Down:    [null]
Other Props:    Object has no additional properties
Children:    Container has no children
Ancestors:    "Account Settings" : grouping : normal
    "Info" : pane : normal
    "" : client : normal
    "Backstage view" : pane : normal
    "" : pane : normal
    "" : window : focused,focusable
    none : client : focusable
    none : window : focusable
    "Mail to Read - Paul Schrader 2012 - Outlook" : client : focusable
    "Mail to Read - Paul Schrader 2012 - Outlook" : window : sizeable,moveable,focusable
    "Desktop" : client : focusable
    "Desktop" : window : focusable
    [ No Parent ]

==================================

SimpleSpy

Mouse position is retrieved 217-247
At least we have an element [Account Settings...][NetUIAnchor]
Having the following values for all properties:
Title is: <Account Settings...>    Class   := <NetUIAnchor>    controltype:= <UIA_MenuItemControlTypeId>    ,<50011>    , (0000C35B)    
*** Parent Information ***
Title is: <Account Settings>    Class   := <NetUISlabContainer>    controltype:= <UIA_GroupControlTypeId>    ,<50026>    , (0000C36A)    
*** Detailed properties of the highlighted element ***
UIA_title:= <Account Settings...>
UIA_text:= <Account Settings...>
UIA_regexptitle:= <Account Settings...>
UIA_class:= <NetUIAnchor>
UIA_regexpclass:= <NetUIAnchor>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_RuntimeId:= <42;1639720;2;963468816;522>
UIA_BoundingRectangle:= <180;207;114;86>
UIA_ProcessId:= <3792>
UIA_ControlType:= <50011>
UIA_LocalizedControlType:= <Menu Item>
UIA_Name:= <Account Settings...>
UIA_AccessKey:= <Alt, F, I, S>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <NetUIAnchor>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <True>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <True>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <0>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Account Settings...>
UIA_LegacyIAccessibleRole:= <12>
UIA_LegacyIAccessibleState:= <1074791424>
UIA_LegacyIAccessibleKeyboardShortcut:= <Alt, F, I, S>
UIA_LegacyIAccessibleDefaultAction:= <Open>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:3792,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:mso.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

 

Edited November 11, 2016 by HighlanderSword

[LarsJ]

It's a menu item. So I think you just need to replace $UIA_ButtonControlTypeId with $UIA_MenuItemControlTypeId (you can find these constants in CUIAutomation2.au3) in the code in post 507. And maybe "Button" with "MenuItem" in $pButton and $oButton.

[HighlanderSword]

Same issue.......

[LarsJ]

Are you using the name "Account Settings..." including the dots?

[HighlanderSword]

Hello,

 

Yes , below is the code I'm using.......

 

 

 

 Local $hWindow = WinGetHandle( "Mail to Read" )
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF )

  ; Condition to find split button

 

  ;find the account settings and click it

  ; Condition to find split button
  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId,  $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Condition to find "Lists"
  ; "Lists" is a localized word
  ; Use the word in your own language
  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Account Settings...", $pCondition2 )
  msgbox(0,"66666",$pcondition2)
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  ; Both conditions
  Local $pCondition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  msgbox(0,"pcon2",$pcondition)
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find "Lists" split button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  msgbox(0,"77777",$pbutton)
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; Click (Invoke) button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_LegacyIAccessibleDefaultActionPropertyId, $pInvoke )

  MsgBox(0,"pinvoke",$pInvoke)
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()

[LarsJ]

You're still getting the "$oButton ERR" which means that it can't find the button or menu item?

Is the file menu open while you're running the code? To click an item ("Account Settings...") in a menu the menu must be open.

In post 509 I can see that UIA_IsInvokePatternAvailable = <False> which probably means that the Invoke method is not working. If Invoke is not working the easiest is to use the MouseClick command.

I can also see that UIA_AccessKey = <Alt, F, I, S>. Using the AccessKey to open the "Account Settings" window could be an easy alternative solution.

The GetCurrentPattern method can only be used with pattern IDs (in top of CUIAutomation2.au3). $oButton.GetCurrentPattern( $UIA_LegacyIAccessibleDefaultActionPropertyId, $pInvoke ) will not work.

I'm not using Office/Outlook so I can't test anything myself. I've Office 2003 installed on my old XP, but I don't think that'll help.

(You can use the "<>"-button to post code or output from the spy tool.)

[junkew]

Which outlook version? which windows version? which autoit version?

Can you try with the example code that the simplespy gives you and post it here.

At first sight it looks a straightforward hierarchy of 2-3 levels deep. Shouldn't be that hard.

Tip: Highlight first before the actual action you want to do so you know your recognition pattern is correct

• indeed frequently invoke doesn't work in the way you expect

its this function in uiawrappers and calls the _UIA_DrawRect function that draws a rectangle on the desktop in the specified area

Func _UIA_Highlight($oElement)
    Local $t
    $t = StringSplit(_UIA_getPropertyValue($oElement, $UIA_BoundingRectanglePropertyId), ";")
    _UIA_DrawRect($t[1], $t[3] + $t[1], $t[2], $t[4] + $t[2])
EndFunc   ;==>_UIA_Highlight

 

[HighlanderSword]

Office Pro Plus 2013

Autoit Ver 3.3.14.2

Try with what code from Simplespy, do you mean highlight the Action I want and the Ctrl W to capture the control details

[junkew]

Start simplespy

Move mouse over your object

Ctlr w

Now you should see output including example source that you copy paste to new autoit script. post then the source and result so we can enhance where needed.

[HighlanderSword]

Hello,

Below is what I get from simplespy, I don't get any Autoit Code

Can you post the SimpleSpy Au3 Code you are using to get code as well ?

 

 

=======================================================================

Mouse position is retrieved 232-238
At least we have an element [Account Settings...][NetUIAnchor]
Having the following values for all properties:
Title is: <Account Settings...>    Class   := <NetUIAnchor>    controltype:= <UIA_MenuItemControlTypeId>    ,<50011>    , (0000C35B)    
*** Parent Information ***
Title is: <Account Settings>    Class   := <NetUISlabContainer>    controltype:= <UIA_GroupControlTypeId>    ,<50026>    , (0000C36A)    
*** Detailed properties of the highlighted element ***
UIA_title:= <Account Settings...>
UIA_text:= <Account Settings...>
UIA_regexptitle:= <Account Settings...>
UIA_class:= <NetUIAnchor>
UIA_regexpclass:= <NetUIAnchor>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_RuntimeId:= <42;8259664;2;1087324544;543>
UIA_BoundingRectangle:= <180;207;114;86>
UIA_ProcessId:= <10264>
UIA_ControlType:= <50011>
UIA_LocalizedControlType:= <Menu Item>
UIA_Name:= <Account Settings...>
UIA_AccessKey:= <Alt, F, I, S>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <NetUIAnchor>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <True>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <True>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <0>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Account Settings...>
UIA_LegacyIAccessibleRole:= <12>
UIA_LegacyIAccessibleState:= <1074791424>
UIA_LegacyIAccessibleKeyboardShortcut:= <Alt, F, I, S>
UIA_LegacyIAccessibleDefaultAction:= <Open>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:10264,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:mso.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

[junkew]

Install from first post latest zip

[Tahir_Mirza]

I am a beginner on Autoit and wanted  to understand UI automation code so the question is where do I start? I already went through some of examples but I am not able to understand the use of functions that are part of UIAutomation is there any documentation where I can start with some basics and using those basic to automate advanced topic. some times it happens that looking at code one cannot understand the arguments passed and the returned object from a function so is there any doc or link to get overcome such problems?

Small examples in UIAutomation (not auto it example code) to perform basic operations like reading data from a list view or tree nodes will be helpful.

After looking at the code and some examples executed by me I have following questions :

What are the steps followed in order to extract rows from a table like view?(for e.g in outlook we have mail list for which UDFs are defined so similarly using UIautomation)

Where should I look for understanding the different UIA properties and their purpose defined in the UIAWrappers?

How do you get the text value from a input textbox using UIAutomation and not autoit example code?

How to set the textbox using UIAutomation(not auto it example code)?

How to identify a table from  a window and get the rows of tables and extract the icons or text in contents of that row?

How do you go to the parent container of that table as when one uses simple spy it directly gives you the autogenerated code to click on that particular element and it does not help to extract the contents of complete rows present in that table?

Sometimes the simplespy code does not identify the element correctly and starts random highlighting and selection so how to correctly identify the UI element?

Thanks in advance.