IUIAutomation MS framework automate chrome, FF, IE, ....

LarsJ · May 18, 2014

Excellent junkew. I will take a closer look at the zip files.

gaogao · May 22, 2014

excellent work, please consider the attached patch that allows the use of the "MustDeclareVars" option.

UIAWrappers.declareVars.diff.zip

dilipped · June 18, 2014

This example prints the items of the Windows Explorer listview in Scite console. The selected state of the items is printed too.

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

Hi,

I used this code to list all the files in a window. But I also want to list corresponding details like ModifiedTime, Size etc along with this. Can you let me know how I can retrieve those information?

LarsJ · June 18, 2014

I certainly can. The subitems with the detail information are children of the row (parent). This function prints the name and value of all children to a parent:

Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

Note that I have copied most of the code.

Call the function like this just before the Next statement

ListChildren( $oUIElement )

Regards Lars.

dilipped · June 18, 2014

I certainly can. The subitems with the detail information are children of the row (parent). This function prints the name and value of all children to a parent:

Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

Note that I have copied most of the code.

Call the function like this just before the Next statement

ListChildren( $oUIElement )

Regards Lars.

Hi LarsJ,

I tried what you said but it just prints the name of the file and nothing else. Is there anything like UIA_NamePropertyId field for modifiedDate and size?

Edited June 18, 2014 by dilipped

LarsJ · June 18, 2014

It works for me on Windows 7. Try the full code:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" ) ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )  ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )

    ListChildren( $oUIElement )
  Next

EndFunc


Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

dilipped · June 18, 2014

It works for me on Windows 7. Try the full code:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" ) ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )  ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )

    ListChildren( $oUIElement )
  Next

EndFunc


Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

Great!! It works. Thanks a ton. I had made some changes while trying something out and forgot to change it back. That's the reason it did not work in the first go.

LarsJ · June 18, 2014

Good, to see. Note that Windows Explorer must be in details view.

dilipped · June 18, 2014

@LarsJ. Actually I started recently with Autoit and though I am comfortable with autoit I struggle a lot with coding using UIAutomation components. It would be helpful if you can show me a roadmap on how to start learning it. Some resources or links to tutorials would be helpful

LarsJ · June 19, 2014

CUI Automation: Documentation Roadmap

Overview

Windows Automation API
UI Automation
UI Automation Fundamentals General concepts
UI Automation Client Programmer's Guide To automate applications, this is for us
~~UI Automation Provider Programmer's Guide~~ To create applications, this is not for us

UI Automation Fundamentals

UI Automation Tree Overview
UI Automation Control Types Overview
UI Automation Properties Overview To get information
UI Automation Control Patterns Overview To perform actions
UI Automation Events Overview To handle events

UI Automation Client Programmer's Guide

Main interfaces (objects)

Other topics

Automating the listview in Windows Explorer

Edited June 19, 2014 by LarsJ

MyEarth · June 21, 2014

Hello guys, i need someone to help me for accomplish a little task

I need to extract text ( just that, not automate ) from a "standard" Windows but this Windows as class DirectUIHWND, i have try many method but nothing work i can't read the text. Seaching for an alternative to Autoit Info Window i have found the "obsolete" UISpy and that was the only one that give me information about that control with the list of text i need to extract

I have use the "Simple spy demo" and i can access the "listview" but only the single "element", this is is what i get:

At least we have an element [name][Element]
Having the following values for all properties: 
Title is: <name>    Class   := <Element>    controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    
*** Parent Information ***
Title is: <listitem>    Class   := <Element>    controltype:= <UIA_ListItemControlTypeId>   ,<50007>    , (0000C357)    
*** Detailed properties of the highlighted element ***

The difference between the first item and the second are ( at left the first, at right the second )

UIA_BoundingRectanglePropertyId :=287;279;211;15 --- UIA_BoundingRectanglePropertyId :=287;327;64;15
UIA_ClickablePointPropertyId :=392;286 --- UIA_ClickablePointPropertyId :=319;334
UIA_HelpTextPropertyId :=Name of the item1 --- UIA_HelpTextPropertyId :=Name of the item2
UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 1 --- UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 2
UIA_LegacyIAccessibleValuePropertyId :=Name of the item 1 --- UIA_LegacyIAccessibleValuePropertyId :=Name of the item 2
UIA_RuntimeIdPropertyId :=1336;138542016;0 --- UIA_RuntimeIdPropertyId :=1336;138543168;0
UIA_ValueValuePropertyId :=Name of the item 1 --- UIA_ValueValuePropertyId :=Name of the item 2

Ovb i need to extract all the name list like:

Name of the item 1

Name of the item 2

etc.

The name can be different and i don't know the number of the elements of the list

A complete list of the first element:

Mouse position is retrieved 416-289
At least we have an element [name][Element]
Having the following values for all properties: 
Title is: <name>    Class   := <Element>    controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    
*** Parent Information ***
Title is: <listitem>    Class   := <Element>    controltype:= <UIA_ListItemControlTypeId>   ,<50007>    , (0000C357)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=listitem_name
UIA_BoundingRectanglePropertyId :=287;279;211;15
UIA_ClassNamePropertyId :=Element
UIA_ClickablePointPropertyId :=392;286
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50020
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=DirectUI
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=Name of the item 1
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=False
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=False
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=True
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=True
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 1
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=name
UIA_LegacyIAccessibleRolePropertyId :=41
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=64
UIA_LegacyIAccessibleValuePropertyId :=Name of the item 1
UIA_LocalizedControlTypePropertyId :=testo
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=name
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=1336
UIA_ProviderDescriptionPropertyId :=[pid:1336,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:DUI70.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=1336;138542016;0
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=Name of the item 1
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

I think i have say it all but if need more information i'll post it. Thanks for the big hand

P.S. Just for know, the UISPy tool say there is a:

list "scrollviewer"

Under that two element:

scrollbar "Vertical"
list "listview"

Under the "listview" there are items called "listitem", so pratically all the list i need

Edited June 21, 2014 by MyEarth

LarsJ · June 21, 2014

Try this. The code should print all listview items:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Get window handle
  Local $hWindow = WinGetHandle( "Write something here to identify your window" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find listview items
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find listview items
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find listview items ERR" & @CRLF )
  ConsoleWrite( "Find listview items OK" & @CRLF )

  ; Print listview items
  Local $pUIElement, $oUIElement, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    ConsoleWrite( $value & @CRLF )
  Next

EndFunc

MyEarth · June 21, 2014

Thanks Lars, i'll try it very soon. I have a question, is true that the "capture" using UIFramework work only if the window is visible? Long time ago i have try one of the example but if the windows was @SW_HIDE or minimized nothing happened. If is true, there is a "workaround" for have the window not visible to the user and "capture" the information? Thanks again

LarsJ · June 21, 2014

It's true, that there are some issues with minimized or hidden windows. If you test the code in post 212 with the listview in Windows Explorer, it'll not print any files/folders, if the window is minimized. I don't know if there is a workaround. May be junkew has a solution.

You can try to do a test with Inspect.exe (in Windows SDK). If Inspect.exe can get the information, it should also be possible to get the information with this UDF.

junkew · June 21, 2014

no solution see

http://stackoverflow.com/questions/19606352/ui-automation-doesnt-see-hidden-windows

offcourse there are better ways to retrieve content of a folder

Skitty · June 26, 2014

This UDF is absolutely amazing...

Is there anyway to extract the HTML document into a variable with this?

junkew · June 26, 2014

no, the only way to do that is to switch to a native component for the specific browsers

yes, indirectly by opening the menu/view to view the html source or navigate thru the development tooling menu with UIA actions

marko001 · July 1, 2014

@junkew , @LarsJ

Great work, in years of work with AutoIt still never tried using IUIAutomation.

I tried to mesh around the code but have still some difficulties on it.

Mainly i'm getting too many errors like:

F:\AUTO-IT Projects\___2.0 __\ex\ex2_Taskbar.au3 (26) : ==> Variable must be of type "Object".:

$oStart.getCurrentPattern($UIA_InvokePatternId, $pInvoke)

$oStart^ ERROR

->11:29:39 AutoIT3.exe ended.rc:1

anyway, I started looking at the code of you first example, spy_demo.au3 and understood what I need to find and where to find.

I need just a "kickstart":

In the spy demo your starting point is the mouse position used as starting point to find/evaluate controls.

    $x = MouseGetPos(0)

    $y = MouseGetPos(1)

    DllStructSetData($tStruct, "x", $x)

    DllStructSetData($tStruct, "y", $y)

    ConsoleWrite(DllStructGetData($tStruct, "x") & DllStructGetData($tStruct, "y"))



;~ consolewrite("Mouse position is retrieved " & @crlf)

    $UIA_oUIAutomation.ElementFromPoint($tStruct, $UIA_pUIElement)

Instead I need to know whether a condition occurs or not and "do things" in case it occurs.

I.e. I want to dismiss a pop-up if it occurs but I don't know where or when it occurs.

Thanks to your spy demo i realized that the control is given by

Mouse position is retrieved 813-427

At least we have an element [Close][Button]

Having the following values for all properties:

Title is: <Close>    Class   := <Button>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    

*** Parent Information ***

Title is: <Shiny.Chat.ViewModels.PrivateChatSessionViewModel>    Class   := <ToolWindow>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    

*** Detailed properties of the highlighted element ***

UIA_AcceleratorKeyPropertyId :=Close

UIA_AccessKeyPropertyId :=

UIA_AriaPropertiesPropertyId :=

UIA_AriaRolePropertyId :=

UIA_AutomationIdPropertyId :=CloseButtom

UIA_BoundingRectanglePropertyId :=802;414;24;24

UIA_ClassNamePropertyId :=Button

UIA_ClickablePointPropertyId :=814;426

what I would like to do is create a function

AdlibRegister("_dismiss", 1000)

that checks the following:

if Class = Button and Title = Close and Title is <Shiny.Chat.ViewModels.PrivateChatSessionViewModel>   then
  close the popup
endif

To close popup i could use

mouseclick("left",UIA_ClickablePointPropertyId :=814;426)

or maybe you can suggest me a solution without mouseclicks.

Thanks, hope you can help me with this.

Best regards,

Marco

Edited July 1, 2014 by marko001

junkew · July 1, 2014

you have to download the zip with the examples. example code in the thread will not work due to some renaming of variables.

Variable must be of type "Object" message you just get when you did not have a valid object based on the search criteria given

There are different ways of doing this (easy and hard)

a. you just have a loop and every nnn milliseconds you check for your popup which is more or less what you are doing

b. use the event examples that tell you when there is a new window appearing examples 19-24

marko001 · July 1, 2014

I always worked with loops, I know it's the easiest way, but honestly not the best from a programmer's point of view.

Can you give me some hints on way?

Thanks a lot mate, meanwhile I'll study the codes.

Marco

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

junkew

junkew

junkew

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Similar Content