IUIAutomation MS framework automate chrome, FF, IE, ....

[Jfish]

@Junkew -

This may be obvious but any help the spy can assist with in identifying the control automatically to the point where it takes some of the complexity away from interpreting the info returned would be huge.  Example, if I Spy the Word app can the Spy figure out the PID and call _ProcessGetName($iPID) to do something like this?:

_UIA_setVar("[RETURNED PROCESS NAME].mainwindow","classname:=[RETURNED CLASS NAME]")

In this example the tool spied everything but might ask me to name the control (i.e. "mainwindow").  I believe the PID would return "Word.exe."  I realize this will not always work because there can be more than one control with the same name but can the Spy search for the instance that responds to the coordinates of the control selected in the Spy and determine which instance it is?  I am just tossing stuff out to see how automated the identification can be.  In an ideal world I highlight a control and the Spy writes the code to call it with little or no input.

[Shrapnel]

I think in my post (# 57 = ) that i was looking for the RTI object in the wrong spot.

junkew, does this make more sense?

func _UIA_action($obj_or_string, $strAction, $p1=0, $p2=0, $p3=0)

    local $tPattern
    local $x, $y
;~  local $objElement
    local $oElement

;~ If we are giving a description then try to make an object first by looking from repository
;~ Otherwise assume an advanced description we should search under one of the previously referenced elements at runtime
    if isobj($obj_or_string) or IsObj(_UIA_getVar("RTI." & $obj_or_string)) Then ; check if obj_or_string is an object, or if it is stored in the RTI
        If isobj($obj_or_string) Then ; if object, use it
            $oElement = $obj_or_string
            $obj = $obj_or_string
        Else ; if RTI.object, use that
            $oElement = _UIA_getVar("RTI." & $obj_or_string)
            $obj = _UIA_getVar("RTI." & $obj_or_string)
        EndIf
    else
; rest of function....

[junkew]

@JFish: I will extend the simple spy with parent information and with a simple editbox where you directly can test the given description by highlighting it once you press a button highlight. Will do that somewhere coming days. A more advanced spy is on the TODO list but not on short notice
 
@Shrapnel:
Technically all objects are in UI Automation tree and within this abstraction framework I make a Run Time Information tree / dictionary for some frequently used objects like the mainwindow. I am a little in doubt if I should add all objects that are used in the RTI tree. Initially I started by only keeping a quick reference to the MAINWINDOW as its a frequent starting point for searching. If I would put all objects in RTI tree I am actually rebuilding the caching of UIA framework itself and thats for sure not what my goal is.

• So to answer your question:

I try to abstract this piece of code but not fully cleared up my mind what should be in RTI and what not. I can also imagine that I want to have much more information in the RTI tree for heuristically finding an object back when the main properties given cannot identify the control.

 
Some directions I want to go 
syntax like this

_UIA_Action("IE.MainWindow","menu","File,Send,Page by Email","click")

 So meaning look under mainwindow for a menu and click sequentially the menu/submenu's on the text as given
But when you need it multilanguage (english and dutch for example) I would like to have defined as a replacing reference logic without touching the _UIA_action call only by adding the descriptions to the RTI (so maintenance primarily is done by maintaining the UI definition map)

File:=((File)|(Bestand))
Send:=((Send)|(Verstuur))
Page by Email:=((Page by Email)|(Pagina per email))

 and even worse the menu titles can be different per browser (IE, Chrome, Firefox, Safari) where I prefer to have the main scripts equal and deal with the details only by giving different definitions or use more complicated regular expressions

 
 Working currently on

• UID files to create a User Interface Definition file which describes the logical and physical pattern
• UIA.CFG file to have the basic settings configurable thru a file (and if not there use some defaults)
• CFG files per user (for multiuser environment)
• extending some actions in the _UIA_Action

 

Edited November 12, 2013 by junkew

[Shrapnel]

Not sure if this would be possible (or worth it), but i think it would be nice if multiple attributes could be defined to a control at one time similar to how controls are handled with Autoit http://www.autoitscript.com/autoit3/docs/intro/controls.htm

 

meaning _UIA_setVar() would be set like this:

_UIA_setVar("OrderTracking.btnNewOrder", "[automationid:=btnNewOrder; class:=WindowsForms10.BUTTON.app.0.267e1d0_r13_ad1]")

and when a function like _UIA_Action is called, both of those attributes would have to match

 

 

Edit: Sorry, i didn't even notice that this was already on the TODO list!

TODO

• Build recorder
• Enhance the spy with a nicer UI
• UI for the repository (now in the script with dot notation)
• Enhance mapping / identifying on multiple properties instead of 1 combined with index
• If speed becomes an issue use the caching logic of the MS UIA framework

Edited November 13, 2013 by Shrapnel

[Bluesmaster]

Fantastic work !

Just one question:

how to get an object from a native window handle without searching throuh all desktop children?

lets say for an explorer listview

hListview=ControlGetHandle($hWin,'','[CLASS:DirectUIHWND]')

I tried:

 $objUIAutomation.ElementFromHandle( $hListview, $pUIElement )
 $oUIElement = objcreateinterface( $pUIElement , $sIID_IUIAutomationElement , $dtagIUIAutomationElement )

but it crashes.

And in Snippet from "trancexxx" I dont know what to place for  "sIID_IAccessible"

Func objFromHandle( $hWindow )

    $tIID_IAccessible = _WinAPI_GUIDFromString( $sIID_IAccessible )

    $aCall  = DllCall($hOLEACC_DLL, "long", "AccessibleObjectFromWindow", "hwnd", $hWindow, "dword", $OBJID_CLIENT, "struct*", $tIID_IAccessible, "ptr*", Null)
    $pIAccessible =  $aCall[4]

    ; Object from pointer
     $oIAccessible = ObjCreateInterface( $pIAccessible , $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

EndFunc

???

Thanks for suggestions

Bluesmaster

Edited November 13, 2013 by Bluesmaster

[junkew]

@Bluesmaster: 
First part I can answer second part probably TranceXXX can explain that in a better way

 $objUIAutomation.ElementFromHandle( $hListview, $pUIElement )  

 
You have to fix in CUIAutomation2. There seems to be some none definitions in parameters that should be  
 

"ElementFromHandle hresult(hwnd;ptr*);" & _

@Shrapnel: 

• Yes multiple properties matching is definitely on my TODO list on relatively short term to fix that 

[junkew]

Updated first post with new wrappers/fixes etc.
 
For this example to work you have to make a file UIA.CFG where you can store some of the data

For identifying controls I try to put in the dutch and english values for propery names. Sometimes its a best guess for english (I test against a dutch windows)
 
Example 11 Word 2010, Calculator, Notepad

expandcollapse popup

#include "UIAWrappers.au3"

#AutoIt3Wrapper_UseX64=N

;~ Start the system under test applications
_UIA_StartSUT("SUT1") ;~Calculator
_UIA_StartSUT("SUT2") ;~Notepad
_UIA_StartSUT("SUT3") ;~MS Word

;~ To be moved to UID (User interface definition) files
;~ Set the system under test UID objects to recognize
local $UID_WORD[4][2] = [ _
["mainwindow","classname:=OpusApp"], _
["btnZoeken","name:=((Zoeken.*)|(Find.*)); ControlType:=Button; acceleratorkey:=Ctrl\+F"] , _
["document","classname:=_WwG"], _
["btnBold","name:=((Vet)|(Bold))"] _
]
_UIA_setVarsFromArray($UID_Word,"Word.")

local $UID_CALC[7][2] = [ _
["mainwindow","classname:=CalcFrame"], _
["1","name:=1; controltype:=button"],  _
["2","name:=2; controltype:=button"] , _
["3","name:=3; controltype:=button"] , _
["BACKSPACE","AutomationId:=83"] , _
["mnuEdit","name:=((Edit)|(Bewerken)); controltype:=MenuItem"], _
["mnuCopy","name:=((Copy.*)|(Kopi.*)); controltype:=MenuItem"] _
]
_UIA_setVarsFromArray($UID_CALC,"Calculator.")

local $UID_NOTEPAD[4][2] = [ _
["mainwindow","classname:=Notepad"], _
["title","controltype:=50037"], _
["mnuEdit","name:=((Edit)|(Bewerken))"], _
["mnuPaste","name:=((Paste.*)|(Plak.*))"] _
]
_UIA_setVarsFromArray($UID_NOTEPAD,"Notepad.")

;~ To be moved to 1 or multiple scriptfiles
;- Set the script
;~ The actual script, 50,10 is only there if a lot of lines and parameters are needed

local $script[50][10]= [ _
["Word.Mainwindow","setfocus"], _
["Word.btnBold",   "click"], _
["word.document",  "sendkeys", "hello world"], _
["word.btnZoeken", "click"],  _
["calculator.mainwindow", "setfocus"], _
["calculator.1","click"], _
["calculator.2","click"], _
["calculator.3","click"], _
["calculator.backspace","click"], _
["calculator.mnuEdit","click"], _
["calculator.mnuCopy","click"], _
["notepad.mainwindow", "setfocus"], _
["notepad.mnuEdit","click"], _
["notepad.mnuPaste","click"], _
["notepad.mainwindow","setvalue", "hello world"] _
]

_UIA_launchScript($script)

Exit

save this to UIA.CFG file

expandcollapse popup

; This is an inifile for UIA wrappers having the configuration defaults
; Debug=true        Turn debugging on of off with true/false value
; Highlight=true    Turn Highlighting rectangle to true / false 
; TODO: AutoStartSUT=true AutoStartSUT is starting all SUT's automatically

[Global]
Debug=true
Highlight=true
AutoStartSUT=true

[Multiuser]
CFGPerUser=false

[Folders]
subfolders=false

;System under test settings
; Folder      = Location where exe can be found
; Workingdir  = Location of the working directory
; exe         = Name of the exe to start
; Fullname    = Path & name of exe 
; Windowstate = minimized, maximized, normal

[SUT1]
Folder=%Windowsdir%\system32
Workingdir=%Windowsdir%\system32
exe=calc.exe
Fullname=%Windowsdir%\system32\calc.exe
Parameters=
Processname=calc.exe
Windowstate=normal

[SUT2]
Folder=%Windowsdir%\system32
Workingdir=%Windowsdir%\system32
exe=notepad.exe
Fullname=%Windowsdir%\system32\notepad.exe
Parameters=
Processname=notepad.exe
Windowstate=normal  

[SUT3]
C:\Program Files (x86)
Folder=%programfilesdir%\Microsoft Office\Office14
Workingdir=%programfilesdir%\Microsoft Office\Office14
exe=winword.exe
Fullname=%programfilesdir%\Microsoft Office\Office14\winword.exe
Parameters=
Processname=winword.exe
Windowstate=normal

[corgano]

This is AMAZING! Please keep working on this! someone needs to UDF this. It's amazing! I especially like the chrome automation, no plugins required! Being able to do IE and Excel and the other stuff is just awesome. I am nowhere near this level of programming, but i hope this makes it to a standard UDF

Good luck, and thankyou

[sri6840]

 Can any one give me a solutions for the below mentioned question

1. Solutions to "wait for page load" on chrome. Similar to _IELoadWait

2. Click on element by ID

These options are much much needed for me.. can any one Solve this.

[junkew]

Not exactly the right place to ask question like this. you should be asking in general support forum

http://www.autoitscript.com/autoit3/docs/functions/MouseGetCursor.htm

most likely chrome changes the cursor when loading a page

[LarsJ]

I needed information about the controls in a Windows Explorer window for a project (Windows Explorer right pane) I'm working on. It was easy to get the information with this CUIAutomation2.au3 UDF.

Here is the code I used. It's very simple. The elements of the Automation tree are extracted with a recursive function. The parent-child relationships of the controls are printed in Scite console in a hierarchical structure like a treeview. This gives a nice overview.

expandcollapse popup

#include <WinAPI.au3>
#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

Example()


; List all automation elements of a window
; in a hierarchical structure like a treeview.
Func Example()
  ; Get window handle
  ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Windows Forms
  ;Local $hWindow = WinGetHandle( "[REGEXPCLASS:WindowsForms10.Window.8.app.*]" )          ; Windows Updates Downloader
  ;Local $hWindow = WinGetHandle( "BCGPVisualStudioGUIDemo - Start Page" )                 ; BCGSoft BCGPVisualStudioGUIDemo
  ;Local $hWindow = WinGetHandle( "[REGEXPCLASS:^(Cabinet|Explore)WClass$]" )              ; Windows Explorer on XP, Vista, 7, 8, 10
  ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" )         ; AutoIt GUI window
  ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" )                                ; AutoIt script
  ;Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" )                           ; Chrome
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                                      ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "Calculator" )                                           ; Calculator
  Local $hWindow = _WinAPI_GetDesktopWindow()                                              ; Desktop. Beware. Can print a lot of information.
  If Not $hWindow Then Return ConsoleWrite( "Window handle error" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object error" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window error" & @CRLF )

  ; List all elements of window
  ListDescendants( $oWindow, 0, 0 )
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )
  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd
EndFunc

Func GetCurrentPropertyValue( $oObject, $iPropertyId )
  Local $vValue, $sString
  $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue )
  If Not IsArray( $vValue ) Then Return $vValue
  $sString = $vValue[0]
  For $i = 1 To UBound( $vValue ) - 1
    $sString &= "; " & $vValue[$i]
  Next
  Return $sString
EndFunc

Tested on Windows 7 64 bit and Windows XP 32 bit. If you are on a 64 bit Windows you must run the example as a 64 bit program.

Note that this example does not handle virtual items in Windows Explorer listview. To handle virtual items see post 94 (example 18 in the list in first post).

Note also that this example is not limited to Windows Explorer. It can print the Automation tree (hierarchical structure of all elements) for any window.

Edited April 25, 2017 by LarsJ
Code updated

[LarsJ]

And of course you can still get some information with the functions in WinAPI.au3:

expandcollapse popup

#include <Constants.au3>
#include <WinAPI.au3>

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ;Local $hWindow = _WinAPI_GetDesktopWindow()                        ; Desktop
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )        ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )       ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane", "" ) ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]", "" )             ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "Calculator" )                      ; Calculator
  If Not $hWindow Then Return

  ;ListDescendants( $hWindow, 0, 1 ) ; Desktop
  ListDescendants( $hWindow, 0, 0 )  ; Window

EndFunc


Func ListDescendants( $hParent, $iLevel, $iLevels = 0 )

  If Not $hParent Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  Local $hChild = _WinAPI_GetWindow( $hParent, $GW_CHILD )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While $hChild
    ConsoleWrite( $sIndent & "Handle = " & Hex( $hChild ) & @CRLF & _
                  $sIndent & "Class  = " & _WinAPI_GetClassName( $hChild ) & @CRLF & _
                  $sIndent & "Title  = " & _WinAPI_GetWindowText( $hChild ) & @CRLF & @CRLF )

    ListDescendants( $hChild, $iLevel + 1, $iLevels )

    $hChild = _WinAPI_GetWindow( $hChild, $GW_HWNDNEXT )
  WEnd

EndFunc

[LarsJ]

The examples in post #71 generates a lot of information. For the examples related to Windows Explorer you might be interested only in data for the listview. This example shows how to limit the data to a named control class which is a descendant of the parent window.

On Windows 7 the class name for the Windows Explorer listview is UIItemsView. On Windows XP it's SysListView32. If you are on Vista or Windows 8 you have to figure out the names yourself by running the code in post #71.

Only the function ListClassDescendants (10 lines) is new code. The rest is copied from post #71.

expandcollapse popup

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )        ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )       ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane", "" ) ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]", "" )             ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "Calculator" )                      ; Calculator
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListClassDescendants( $oWindow, "UIItemsView" )    ; Windows Explorer, Windows 7
  ;ListClassDescendants( $oWindow, "SysListView32" ) ; Windows Explorer, Windows XP

EndFunc


Func ListClassDescendants( $oWindow, $sClass )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, $sClass, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElement, $oUIElement
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oUIElement ) Then Return

  ListDescendants( $oUIElement, 0, 1 )

EndFunc


; Same function as in post #71
Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )

  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  Local $pRawWalker, $oRawWalker
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )

  Local $pUIElement, $oUIElement
  $oRawWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title    = " & _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Selected = " & _UIA_getPropertyValue( $oUIElement, $UIA_SelectionItemIsSelectedPropertyId ) & @CRLF & _
                  $sIndent & "Class    = " & _UIA_getPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Type     = " & _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Handle   = " & Hex( _UIA_getPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd

EndFunc


; Same function as in post #71
Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

Here is an example of the output on Windows 7:

Title    = Header
Selected = False
Class    = UIViewHeader
Type     = header
Handle   = 00000000

Title    = Example n1.au3
Selected = True
Class    = UIItem
Type     = list item
Handle   = 00000000

Title    = Example n1-a.au3
Selected = False
Class    = UIItem
Type     = list item
Handle   = 00000000

Title    = zLink
Selected = False
Class    = UIItem
Type     = list item
Handle   = 00000000

Tested on Windows 7 64 bit and Windows XP 32 bit. If you are on a 64 bit Windows you must run the example as a 64 bit program.

Note that this example does not handle virtual items in Windows Explorer listview. To handle virtual items see post 94 (example 18 in the list in first post).

Note also that this example is not limited to Windows Explorer. It can print the descendants of an arbitrary named control class.

Last edit: Notes.

Edited July 21, 2014 by LarsJ

[junkew]

See also findall method

example in UIAWrappers

It will give you directly an array of all elements (no need to find yourself recursively)

http://msdn.microsoft.com/en-us/library/windows/desktop/ee696027(v=vs.85).aspx

[LOULOU]

Has someone an example for controlling Silverlight object ?

Thanks in advance 

[junkew]

use the simple spy and hover over the silverlight object then you see the details to recognize on.

If you do not understand post the information the spy gives and I can give an example

[LarsJ]

junkew, I'm using the recursive procedure to print the parent-child relationships of the controls in a hierarchical structure like a treeview. This gives a nice overview. That's why I'm not using FindAll.

[LarsJ]

This example prints the items of the Windows Explorer listview in Scite console. The selected state of the items is printed too.

expandcollapse popup

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

Note that this example does not handle virtual items. To handle virtual items see post 94 (example 18 in the list in first post).

Last edit: Note.

Edited July 21, 2014 by LarsJ

[LarsJ]

The information I needed in post #71 was a chain of controls from Windows Explorer top window to the listview. The code in post #71 prints a lot of information. To get this chain for more folders it would be advantageous to be able to print the chain directly. This can be done if you start to find the listview and move upwards through the parent controls to the top window:

 

expandcollapse popup

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  ListParents( $pWindow, 50008 ) ; 50008 = SysListView32 on XP ; 50008 = UIItemsView on 7
  ; Run the code in post #71 to get this value

EndFunc


Func ListParents( $pWindow, $iCtrlType )

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIParent, $aUIParents[20], $iUIParents = 0
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pUIParent )
  If Not $pUIParent Then Return

  $aUIParents[$iUIParents] = $pUIParent
  $iUIParents += 1

  Local $pRawWalker, $oRawWalker, $same
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawWalker ) Then Return

  While Not $same
    $oRawWalker.GetParentElement( $pUIParent, $pUIParent )
    $oUIAutomation.CompareElements( $pWindow, $pUIParent, $same )
    $aUIParents[$iUIParents] = $pUIParent
    $iUIParents += 1
  WEnd

  Local $oUIParent, $sClass, $sIndent = ""
  For $i = $iUIParents - 1 To 0 Step -1
    $oUIParent = ObjCreateInterface( $aUIParents[$i], $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oUIParent.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sClass )
    ConsoleWrite( $sIndent & $sClass & @CRLF )
    $sIndent &= "    "
  Next

EndFunc

That was my first test of this UDF. Not the last. It seems to be very good.

Documentation:

UI Automation Client Programmer's Guide

How-To Topics for UI Automation Clients

IUIAutomation interface

IUIAutomationElement interface

IUIAutomationElementArray interface

IUIAutomationTreeWalker interface

[junkew]

regarding silverlight controls I tested against this on most standard controls just work fine

http://samples.msdn.microsoft.com/Silverlight/SampleBrowser/#/?sref=ACBCustomStyles