UIA Need to speed this code up

[careca]

Only for windows 10. Somehow it just takes too long to take actions. Only in the UIA bit obviously, the rest is fast enough.

expandcollapse popup

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)
Local $sText1, $Vis
ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
            WinWaitActive("Set Associations", 'Set Associations', 10)
            Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
            If $hWnd <> 0 Or @error <> 0 Then
                Do
                    $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
                Until $Vis = 1
                Sleep(100)
                Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
                For $k = 0 To $iCount - 1
                    $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
                    If $sText1 = '.mp3' Then
                        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
                        ConsoleWrite('=============================================================================' & @CRLF)
                        ConsoleWrite('Line - ' & $k & ' - ' & $sText1 & @CRLF)
                        ConsoleWrite('=============================================================================' & @CRLF)
                        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
                        Elem()
                        ExitLoop
                    EndIf
                Next
            EndIf
Func Elem()
Local $oUIElement=_UIA_getObjectByFindAll("Linedown.mainwindow", "title:=Line down;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton")
_UIA_action("oUIElement","click")
MouseWheel ( "down", 100)
_UIA_setVar("oUIElement","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")
_UIA_action("oUIElement","click")
_UIA_setVar("oUIElement","Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton")
_UIA_action("oUIElement","click")
MouseWheel ( "down", 100)
_UIA_setVar("oUIElement","Title:=Look for another app on this PC;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")
_UIA_action("oUIElement","click")
EndFunc

 

UIAWrappers.au3

CUIAutomation2.au3

[junkew]

Most likely your pattern is mismatching. The algorithm then tries to find in different ways to recover. Check the logfile and attach it to this thread.

Also try to find in children and not in full subtree.  For speed its better to find step by step.

 

[TheDcoder]

Also considering using real variables instead of _UIA_setVar, last time I checked it was using an array to store the variables, it will add a lot of overhead if you have a bunch of them.

[junkew]

I never faced speed issues due to the array logic that is used as normally the array is not bigger then a couple of hundred to maybe a few thousands if you really have many screenelements in it.

The array logic I use when you have dozens of screens and you want to do more with documentation generating directly out of the array then use an applicationmap/objectrepository logic like below (which then indirectly i load into the $UIA_Vars which is an array as maps did not work properly and similar issues with dictionary object. Most likely I will switch it to .NET collection or sorted array with .NET au3 lib

Example properties file I use (which is not in uiawrappers)  to do more than only the execution of the script (balance between speed and flexibility). So for example with bmppicdata I can show (offline when application is not active) the snapshot made

screen1.section1.element1 := id=help
screen1.section1.element1.bmppicdata := 4D42.....
screen1.section1.element2 := name=txtUsername
screen1.section1.element1 := xpath=...

@careca Is there a need to do it thru gui or is this way also fine in powershell?

AssociateFileExtensions -FileExtensions .txt -OpenAppPath "C:\Windows\System32\notepad.exe"

Back to topic ;-) 

Step 1. With simplespy try the default code (mine below is in dutch). I spy on the first line with file association .123
  deep find in subtree is probably what you see either in console or in the logfile which basically means: Find a better way

Step 2. Exclude the unneeded parts with some trial/error by commenting/uncommenting

    when it highlights uncomment the setfocus. When it doesn't highlight the line (both highlight/setfocus) is not needed
So the dutch version to find the  first line you are interested in (or use the oP10 parent object for the whole list to investigate the whole list but for that you have to study the virtual list view examples (or just send keydown and getactiveelement)

expandcollapse popup

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1.mainwindow","Title:=Koppelingen instellen;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass")  ;Koppelingen instellen
;~ _UIA_setVar("oP2","Title:=Koppelingen instellen;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass")  ;Koppelingen instellen
;~ _UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName")   ;
;~ _UIA_setVar("oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView") ;Deelvenster Verkenner
;~ _UIA_setVar("oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Deelvenster Mapindeling
;~ _UIA_setVar("oP6","Title:=;controltype:=UIA_PaneControlTypeId;class:=XElement")  ;
;~ _UIA_setVar("oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby") ;
;~ _UIA_setVar("oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage")   ;Een bestandstype of protocol aan een specifiek programma koppelen
;~ _UIA_setVar("oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Een bestandstype of protocol aan een specifiek programma koppelen
_UIA_setVar("oP10","Title:=Lijst met extensies;controltype:=UIA_ListControlTypeId;class:=CCListViewEx") ;Lijst met extensies
_UIA_setVar("oP11","Title:=Extensies;controltype:=UIA_GroupControlTypeId;class:=")  ;Extensies
;~ _UIA_setVar("oP12","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=") ;.123

;~ $oUIElement=_UIA_getObjectByFindAll(".123.mainwindow", "title:=.123;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=.123;controltype:=UIA_TextControlTypeId;class:=") ;ControlType:=UIA_TextControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles

;~ _UIA_Action("oP1.mainwindow","highlight")
_UIA_Action("oP1.mainwindow","setfocus")
;~ _UIA_Action("oP2","highlight")
;~ _UIA_Action("oP2","setfocus")
;~ _UIA_Action("oP3","highlight")
;~ _UIA_Action("oP3","setfocus")
;~ _UIA_Action("oP4","highlight")
;~ _UIA_Action("oP4","setfocus")
;~ _UIA_Action("oP5","highlight")
;~ _UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
;~ _UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
;~ _UIA_Action("oP7","setfocus")
;~_UIA_Action("oP8","highlight")
;~ _UIA_Action("oP8","setfocus")
;~_UIA_Action("oP9","highlight")
;~ _UIA_Action("oP9","setfocus")

local $oListView=_UIA_Action("oP10","getobject")

_UIA_Action($oListView,"highlight")
_UIA_Action($oListView,"setfocus")

_UIA_setVar("RTI.SEARCHCONTEXT", $oListView)

local $oListGroup=_UIA_Action("oP11","getobject")
_UIA_Action($oListGroup,"highlight")
_UIA_Action($oListGroup,"setfocus")

;~ consolewrite(_UIA_getAllPropertyValues($oListView))

;~ _UIA_Action("oP11","highlight") ;commented as using getobject in uiaction
;~ _UIA_Action("oP11","setfocus")  ;commented as using getobject in uiaction
;~_UIA_Action("oP12","highlight")
;~ _UIA_Action("oP12","setfocus")

;~ _UIA_action("oUIElement","highlight")
;~ _UIA_action("oUIElement","setfocus")
;~ _UIA_action("oUIElement","click")



;~ Approach 1
;~ Go to the first in the list, not really needed
send("^{HOME}")

for $i=1 to 5
    send("{DOWN}")
    ;~ Check for the focused Element
    $UIA_oUIAutomation.GetFocusedElement($UIA_pUIElement )
    ;~ consolewrite("Element from point is passed, trying to convert to object ")
    local $oUIElement = objcreateinterface($UIA_pUIElement,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    consolewrite(_UIA_getAllPropertyValues($oUIElement))
Next

;~ Approach 2
;~ Determine some information on the listview
local $pElements
local $arrLength
$oListGroup.FindAll($treeScope_children, $UIA_oTRUECondition, $pElements)
local $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
;~  If there are ** NO ** childs found then there is nothing to search
If _UIA_IsElement($oAutomationElementArray) Then
;~ All elements to inspect are in this array assign the lenght of the array to variable
    $oAutomationElementArray.Length($arrLength)
    consolewrite($arrLength & " items found in the listview" & @CRLF)
Else
    consolewrite("FATAL: unexpected no childs found" & @CRLF)
    $arrLength = 0
EndIf

;~  For $i = 0 To $iLength - 1; it's zero based
for $i=0 to 50
    $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
    $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    local $tmpStr = "Title is: <" & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & ">" & @TAB _
                 & "Class   := <" & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & ">" & @TAB _
                 & "controltype:= " _
                 & "<" & _UIA_getControlName(_UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId)) & ">" & @TAB _
                 & ",<" & _UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId) & ">" & @TAB _
                 & ", (" & Hex(_UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId)) & ")" & @TAB _
                 & ", acceleratorkey:= <" & _UIA_getPropertyValue($oUIElement, $UIA_AcceleratorKeyPropertyId) & ">" & @TAB _
                 & ", automationid:= <" & _UIA_getPropertyValue($oUIElement, $UIA_AutomationIdPropertyId) & ">" & @TAB
    consolewrite($tmpStr & @CRLF)
Next

 

Above you should redo for the english version. To my surprise the 2nd approach gives a different sort order in the retrieved array then is shown on screen.

Edited December 10, 2017 by junkew

[careca]

The log

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171210-192132925"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="5" timestamp="20171210-192137442"> _UIA_normalizeExpression title:=Line down;ControlType:=UIA_ButtonControlTypeId;elements 1-2in properties array</logline>
<logline level="5" timestamp="20171210-192137442">   _UIA_getObjectByFindAll property 1 title:=Line down</logline>
<logline level="5" timestamp="20171210-192137443">  name:[title] value:[Line down] having index 3</logline>
<logline level="5" timestamp="20171210-192137443">   _UIA_getObjectByFindAll property 2 ControlType:=UIA_ButtonControlTypeId</logline>
<logline level="5" timestamp="20171210-192137444">  name:[ControlType] value:[50000] having index 15</logline>
<logline level="5" timestamp="20171210-192137446"> ***** FATAL:???? _UIA_getObjectByFindAll no childarray found for object with following details *****</logline>
<logline level="5" timestamp="20171210-192137460"> UIA_title:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_text:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_regexptitle:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_class:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_regexpclass:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_iaccessiblevalue:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_iaccessiblechildId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_id:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_handle:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RuntimeId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_BoundingRectangle:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ProcessId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ControlType:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LocalizedControlType:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_Name:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_AcceleratorKey:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_AccessKey:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_HasKeyboardFocus:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsKeyboardFocusable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsEnabled:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_AutomationId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ClassName:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_HelpText:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ClickablePoint:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_Culture:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsControlElement:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsContentElement:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LabeledBy:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsPassword:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_NativeWindowHandle:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ItemType:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsOffscreen:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_Orientation:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_FrameworkId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsRequiredForForm:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ItemStatus:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsDockPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsExpandCollapsePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsGridItemPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsGridPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsInvokePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsMultipleViewPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsRangeValuePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsScrollPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsScrollItemPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsSelectionItemPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsSelectionPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsTablePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsTableItemPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsTextPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsTogglePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsTransformPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsValuePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsWindowPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ValueValue:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ValueIsReadOnly:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueValue:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueIsReadOnly:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueMinimum:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueMaximum:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueLargeChange:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_RangeValueSmallChange:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollHorizontalScrollPercent:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollHorizontalViewSize:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollVerticalScrollPercent:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollVerticalViewSize:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollHorizontallyScrollable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ScrollVerticallyScrollable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_SelectionSelection:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_SelectionCanSelectMultiple:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_SelectionIsSelectionRequired:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridRowCount:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridColumnCount:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridItemRow:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridItemColumn:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridItemRowSpan:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridItemColumnSpan:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_GridItemContainingGrid:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_DockDockPosition:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ExpandCollapseExpandCollapseState:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_MultipleViewCurrentView:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_MultipleViewSupportedViews:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowCanMaximize:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowCanMinimize:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowWindowVisualState:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowWindowInteractionState:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowIsModal:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_WindowIsTopmost:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_SelectionItemIsSelected:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_SelectionItemSelectionContainer:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TableRowHeaders:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TableColumnHeaders:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TableRowOrColumnMajor:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TableItemRowHeaderItems:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TableItemColumnHeaderItems:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ToggleToggleState:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TransformCanMove:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TransformCanResize:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_TransformCanRotate:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsLegacyIAccessiblePatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleChildId:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleName:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleValue:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleDescription:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleRole:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleState:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleHelp:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleKeyboardShortcut:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleSelection:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_LegacyIAccessibleDefaultAction:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_AriaRole:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_AriaProperties:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsDataValidForForm:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ControllerFor:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_DescribedBy:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_FlowsTo:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_ProviderDescription:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsItemContainerPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsVirtualizedItemPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
UIA_IsSynchronizedInputPatternAvailable:= &lt;** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **&gt;
</logline>
<logline level="0" timestamp="20171210-192137460"> _UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0</logline>
<logline level="5" timestamp="20171210-192137461"> <propertymatching></propertymatching></logline>
<logline level="0" timestamp="20171210-192137461"> </logline>
<logline level="0" timestamp="20171210-192142812"> _UIA_action: No RTI.SEARCHCONTEXT used for oUIElement
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object oUIElement object a:=0 under RTI.MAINWINDOW object b:=1
  looking for Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton
_UIA_normalizeExpression Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Line down
 name:[Title] value:[Line down] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_ButtonControlTypeId
 name:[controltype] value:[50000] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchRepeatButton
 name:[class] value:[TouchRepeatButton] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 10&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton
_UIA_normalizeExpression Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Line down
 name:[Title] value:[Line down] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_ButtonControlTypeId
 name:[controltype] value:[50000] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchRepeatButton
 name:[class] value:[TouchRepeatButton] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 3336Found match with element: 8 the Name is: &lt;Title is: &lt;Line down&gt;    Class   := &lt;TouchRepeatButton&gt;    controltype:= &lt;UIA_ButtonControlTypeId&gt;    ,&lt;50000&gt;    , (0000C350)    rect := &lt; 1454;690;17;33&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;SmallIncrement&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT Vertical
&lt;action&gt;Action 1 click on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171210-192144203"> </logline>
<logline level="0" timestamp="20171210-192149410"> _UIA_action: No RTI.SEARCHCONTEXT used for oUIElement
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object oUIElement object a:=0 under RTI.MAINWINDOW object b:=1
  looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 12&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 3336Found match with element: 31 the Name is: &lt;Title is: &lt;More apps&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;673;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;OptionsButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 2 click on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171210-192149674"> </logline>
<logline level="0" timestamp="20171210-192155068"> _UIA_action: No RTI.SEARCHCONTEXT used for oUIElement
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object oUIElement object a:=0 under RTI.MAINWINDOW object b:=1
  looking for Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton
_UIA_normalizeExpression Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Line down
 name:[Title] value:[Line down] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_ButtonControlTypeId
 name:[controltype] value:[50000] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchRepeatButton
 name:[class] value:[TouchRepeatButton] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 12&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton
_UIA_normalizeExpression Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Line down
 name:[Title] value:[Line down] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_ButtonControlTypeId
 name:[controltype] value:[50000] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchRepeatButton
 name:[class] value:[TouchRepeatButton] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 3364Found match with element: 8 the Name is: &lt;Title is: &lt;Line down&gt;    Class   := &lt;TouchRepeatButton&gt;    controltype:= &lt;UIA_ButtonControlTypeId&gt;    ,&lt;50000&gt;    , (0000C350)    rect := &lt; 1454;690;17;33&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;SmallIncrement&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT Vertical
&lt;action&gt;Action 3 click on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171210-192156462"> </logline>
<logline level="0" timestamp="20171210-192201683"> _UIA_action: No RTI.SEARCHCONTEXT used for oUIElement
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object oUIElement object a:=0 under RTI.MAINWINDOW object b:=1
  looking for Title:=Look for another app on this PC;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=Look for another app on this PC;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Look for another app on this PC
 name:[Title] value:[Look for another app on this PC] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 12&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=Look for another app on this PC;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=Look for another app on this PC;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=Look for another app on this PC
 name:[Title] value:[Look for another app on this PC] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 3358Found match with element: 59 the Name is: &lt;Title is: &lt;Look for another app on this PC&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;673;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;FindInPCButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 4 click on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
</log>

 

 

[junkew]

@careca just crossposting see my previous reply with example on how i made a solution fully with uiautomation (except not the button but you seemed to manage that one)

Your log shows exactly the problem:   NO PROPERTYVALUE DUE TO NONEXISTING OBJECT 

The most difficult part is to make a unique description. My simplespy is certainly not 100%  delivering unique descriptions as there are to many variations to handle.

In previous example this statement can help to limit the searchcontext (once you know its doing a deep find this will help in speed as it will not start searching all (Grand)parents up till the desktop)

_UIA_setVar("RTI.SEARCHCONTEXT", $oListView)

 

[Danyfirex]

Hello I know you're asking for another thing. But You can probably speed of the first part too to avoid the pageFileAssoc and the listview loop part too.

 

You can create a dummy file with the extension you want (".mp3").

Show the Properties Windows.

Click the change progam button.

handle the UI window.

 

Saludos

 

 

[Earthshine]

Danyfirex your avitar looks like Princess Robot Bubblegum in GTA 5 😂 awesome. I love the TV in that game 

Edited December 10, 2017 by Earthshine

[junkew]

Some more background on how the GUI object hierarchy can look like for understanding where you can have speed issues.

• Simplespy of its own uses the raw view which will not allways nicely work with findall although its still unclear to me what hierarchy findall is using

In my simplespy I tweak a little between the 3 hierarchy models

;~ Walk thru the tree with a treewalker
    $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
;~  $UIA_oUIAutomation.ContentViewWalker($UIA_pTW)
;~  $UIA_oUIAutomation.ControlViewWalker($UIA_pTW)

By commenting-uncommenting it in the simplespy this is the output I get for the hierarchy and it looks more logical to use the Contentviewer hierarchy.
Alternatives for spying are inspect.exe or VisualUIAVerifyNative.exe in microsoft SDK (see faq 31 for all spytools) but then you have to write the property string manually yourself.

;~ $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
_UIA_setVar("oP1","Title:=Koppelingen instellen;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass") ;Koppelingen instellen
_UIA_setVar("oP2","Title:=Koppelingen instellen;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass") ;Koppelingen instellen
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName")  ;
_UIA_setVar("oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView")    ;Deelvenster Verkenner
_UIA_setVar("oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")   ;Deelvenster Mapindeling
_UIA_setVar("oP6","Title:=;controltype:=UIA_PaneControlTypeId;class:=XElement") ;
_UIA_setVar("oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby")    ;
_UIA_setVar("oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage")  ;Een bestandstype of protocol aan een specifiek programma koppelen
_UIA_setVar("oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element") ;Een bestandstype of protocol aan een specifiek programma koppelen
_UIA_setVar("oP10","Title:=Lijst met extensies;controltype:=UIA_ListControlTypeId;class:=CCListViewEx") ;Lijst met extensies
_UIA_setVar("oP11","Title:=Extensies;controltype:=UIA_GroupControlTypeId;class:=")  ;Extensies
_UIA_setVar("oP12","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=")    ;.123

;~  $UIA_oUIAutomation.ControlViewWalker($UIA_pTW)
_UIA_setVar("oP1","Title:=Koppelingen instellen;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass") ;Koppelingen instellen
_UIA_setVar("oP2","Title:=Koppelingen instellen;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass") ;Koppelingen instellen
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName")  ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_PaneControlTypeId;class:=XElement") ;
_UIA_setVar("oP5","Title:=Lijst met extensies;controltype:=UIA_ListControlTypeId;class:=CCListViewEx")  ;Lijst met extensies
_UIA_setVar("oP6","Title:=Extensies;controltype:=UIA_GroupControlTypeId;class:=")   ;Extensies
_UIA_setVar("oP7","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=") ;.123

;~ $UIA_oUIAutomation.ContentViewWalker($UIA_pTW)
_UIA_setVar("oP1","Title:=Koppelingen instellen;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass") ;Koppelingen instellen
_UIA_setVar("oP2","Title:=Koppelingen instellen;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass") ;Koppelingen instellen
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName")  ;
_UIA_setVar("oP4","Title:=Lijst met extensies;controltype:=UIA_ListControlTypeId;class:=CCListViewEx")  ;Lijst met extensies
_UIA_setVar("oP5","Title:=Extensies;controltype:=UIA_GroupControlTypeId;class:=")   ;Extensies
_UIA_setVar("oP6","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=") ;.123

And frequently I accept the subtree search by only defining the mainwindow and leave out the subhierarchy definitions, FAST enough and EASIER in maintenance

potentially this leads to duplicates as same name exist(s) in different parts of the window which I resolve with indexrelative:=##

;~ $UIA_oUIAutomation.ContentViewWalker($UIA_pTW)
_UIA_setVar("oP1.mainwindow","Title:=Koppelingen instellen;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass")  ;Koppelingen instellen
_UIA_setVar("oP6","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=") ;.123

• My intention in a future version of the wrappers is to have

1. some kind of Runtime Type Information so action("ElementName", "setfocus") and action("ElementName", "click") do not lead to 2 deep searches.
   Partly its already in the UIAWrappers as it adds when found another reference "RTI.ElementName" to the array, just adds a prefix but somehow its not working
Workaround was given above with RTI.SearchContext

2. More stable fallback search algorithm when initial object description is not found

3. a better readable log (current xml is harder then previous straightforward 1 liners to a flat log.txt)

 

[careca]

Hey there, there is a lot to process there.

The powershell idea is good and might be enough, i will have to try it out and see if/how i can use it

from the autoit script because some things i do dont work unless i call powershell as admin.

Im interested in knowing more about this UIA method and where i screwed up.

From what i understand i didn't give enough parameters for the code to search the element faster, it fails and searches in all.

But now im overwelmed, i will begin by trying your first method remove the commented lines and try to understand what you did.

 

 

 

[junkew]

Sorry for overwhelming but you picked a nice exe file where all challenges will be around for automation of a GUI including controls not visible to au3inf.exe tool and inspect.exe will give you also a tough time getting the info whereas simplespy will help you much better.

Flow

1. Main window with a list to select from.
   Relatively straight forward from simplespy code given except to leave out some elements of the hierarchy

2. Select some element in the list with either keydown or iterating over the list (approach 1 or 2 as given in post 4)

3. Once a certain element is selected click button (in this case with a regular expression hopefully matches english and dutch directly

;And click the button
_UIA_Action("oP1.mainwindow","setfocus")
_UIA_setVar("oButtonAssociate","title:=.*programm.*;classname:=CCPushButton")
;~ _UIA_Action("oButtonAssociate","highlight")
_UIA_Action("oButtonAssociate","click")

4. Now you have a flyout popup that completely has a different hierarchy in the popup

_UIA_setVar("oP1Popup.MainWindow","Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")   ;Hoe wilt u .3gp-bestanden vanaf nu openen?
_UIA_setVar("oP2","Title:=Flyoutvenster;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")   ;Flyoutvenster
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=Element")  ;
_UIA_setVar("oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element") ;Immersive Openwith Flyout
_UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer")    ;
_UIA_setVar("oP6","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemListContainer") ;
_UIA_setVar("oP7","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemList")

5. So in the end this will be the solution (add it to end of source of post 4) to associate a different program (left in the initial output of the spy

• Observe the naming should be different compared the first main window that was defined
• Observe the regular expressions used
• Observe the opopup.ouielement to also differentiate from other defined ouielement we acted on (all elements I first give a unique name
• Its perfectly allowed to do this

  _UIA_Action("Title:=Films.*;controltype:=UIA_ListItemControlTypeId;class:=AppListTileElement","setfocus")

  
  But my preference is the array in between with "logical" names

;And click the button
_UIA_Action("oP1.mainwindow","setfocus")
_UIA_setVar("oButtonAssociate","title:=.*programm.*;classname:=CCPushButton")
;~ _UIA_Action("oButtonAssociate","highlight")
_UIA_Action("oButtonAssociate","click")

;Define the popup elements
_UIA_setVar("oP1Popup.MainWindow","Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")   ;Hoe wilt u .3gp-bestanden vanaf nu openen?
;~ _UIA_setVar("oP1","Title:=Hoe wilt u .3gp-bestanden vanaf nu openen?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")  ;Hoe wilt u .3gp-bestanden vanaf nu openen?
;~ _UIA_setVar("oP2","Title:=Flyoutvenster;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")    ;Flyoutvenster
;~ _UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=Element")   ;
;~ _UIA_setVar("oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Immersive Openwith Flyout
;~ _UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer") ;
;~ _UIA_setVar("oP6","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemListContainer")  ;
;~ _UIA_setVar("oP7","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemList")   ;
_UIA_setVar("oPopup.oUIElement","Title:=Films.*;controltype:=UIA_ListItemControlTypeId;class:=AppListTileElement") ;ControlType:=UIA_ListItemControlTypeId;classname:=AppListTileElement")

;Do the actions in the popup
_UIA_Action("oP1Popup.MainWindow","setfocus")
_UIA_Action("oPopup.oUIElement","highlight")
;~ _UIA_Action("oPopup.oUIElement","click")

 

Edited December 10, 2017 by junkew

[careca]

Hi, im really strugling with this, the powershell isn't doing it for me, the only method i find that works flawlessly is through Windows GUI.

So im trying to take this beast on, but im finding some difficulties, one is the titles and stuff have different names as expected since @junkew's UI language is different.

You did apply some wildcards from what i understand but for instance, what does this line refer to?

_UIA_setVar("oButtonAssociate","title:=.*programm.*;classname:=CCPushButton")

Can you help me break this down line by line? Is this the "set associations" window, or does it refer to the popup?

I just can't find the CCpushButton anywhere, and no window contains "program" as title.

Im sorry about this, im probably making it harder than it really is.

[junkew]

_UIA_setVar("oButtonAssociate","title:=.*programm.*;classname:=CCPushButton")

last part refers to a button as classname states ;-)

.*refers to any character multiple times so anything can match as long as programm is there in between and 99% sure as it also is classname ccpushbutton a double match is not likely. If you would remove the classname it could also match the link, window etc.

just missed my guess for english by the title one m to many. You can use Change program.* for title, not sure how the three dots would map. Could be just ...

 

 

[junkew]

1. simplespy on .386 line and post the maintainable part of code
2. simplespy on Change program button and post the maintainable part of code

Based on the parent hierarchy I can tell you which ones you need and how your code can look like

 

[careca]

Change program button is detectable with the info tool so thats all  good, the part i need is the rest, the popup and from there on.

Sometimes the filetype has more than a few applications that can be associated, so the "more apps" hyperlink that is on the bottom of the list is hidden.

In my code, i press the scrollbar down button, and then wheel mouse scroll down to get to the end, in your opinion, does the "more apps" part need to be visible?

This is the button of the scrollbar:

Spoiler

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=How do you want to open .htm files from now on?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")    ;How do you want to open .htm files from now on?
_UIA_setVar("oP2","Title:=Flyout window;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")    ;Flyout window
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=Element")    ;
_UIA_setVar("oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Immersive Openwith Flyout
_UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer")    ;
_UIA_setVar("oP6","Title:=Vertical;controltype:=UIA_ScrollBarControlTypeId;class:=TouchScrollBar")    ;Vertical

;~ $oUIElement=_UIA_getObjectByFindAll("Linedown.mainwindow", "title:=Line down;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Line down;controltype:=UIA_ButtonControlTypeId;class:=TouchRepeatButton") ;ControlType:=UIA_ButtonControlTypeId;classname:=TouchRepeatButton")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")

This is the more apps hyperlink:

Spoiler

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=How do you want to open .htm files from now on?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")    ;How do you want to open .htm files from now on?
_UIA_setVar("oP2","Title:=Flyout window;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")    ;Flyout window
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=Element")    ;
_UIA_setVar("oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Immersive Openwith Flyout
_UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer")    ;
_UIA_setVar("oP6","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemListContainer")    ;
_UIA_setVar("oP7","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemList")    ;

;~ $oUIElement=_UIA_getObjectByFindAll("Moreapps.mainwindow", "title:=More apps;ControlType:=UIA_HyperlinkControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;ControlType:=UIA_HyperlinkControlTypeId;classname:=TouchHyperLink")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
_UIA_Action("oP7","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")

 

 

Edited December 11, 2017 by careca

[junkew]

There are multiple ways for your problem with powershell but as you said you wanted to learn UIA here some more parts to control your elements thru the GUI interaction

• try post 4 code

;~ Go to the first in the list, not really needed
send("^{HOME}")

but you could go to the end and then whole list is loaded ;-)

;~ Go to the first in the list, not really needed
send("^{END}")

• This part

;~ $oUIElement=_UIA_getObjectByFindAll("Moreapps.mainwindow", "title:=More apps;ControlType:=UIA_HyperlinkControlTypeId", $treescope_subtree)

is better written if needed without .mainwindow as it will influence the search context (in this case will have negative impact), not sure either if invisible elements are found with using findall you can determine that with approach 2 in post 4. comment the for I=0 to 50  and uncomment the one before so loop will iterate all in the list (not sure in my system it has over 1500 extensions ;-))

 $oUIElement=_UIA_getObjectByFindAll("Moreapps.mainwindow", "title:=More apps;ControlType:=UIA_HyperlinkControlTypeId", $treescope_subtree)

same applies for linedown.mainwindow

• this could work for the more choice in the choicelist but check post 4 and 11 for other code snippets to study

;- Make sure mainwindow is around
_UIA_setVar("oP1.mainwindow","Title:=How do you want to open .htm files from now on?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")    ;How do you want to open .htm files from now on?

_UIA_setVar("oUIElement","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;ControlType:=UIA_HyperlinkControlTypeId;classname:=TouchHyperLink")
;-The parent window actions
_UIA_Action("oP1.mainwindow","highlight")
_UIA_Action("oP1.mainwindow","setfocus")
;-Act on a (grand)child of the mainwindow
_UIA_Action("oUIElement","highlight")
_UIA_Action("oUIElement","setfocus")
_UIA_Action("oUIElement","highlight") ;- In case it does not set focus we can see what happens
_UIA_Action("oUIElement","click")
;- or (in future versions above will work like below and I prefer above syntax 
;- $oUIElement=_UIA_Action("oUIElement","getobject")
;- _UIA_Action($oUIElement,"setfocus")
;- _UIA_Action($oUIElement,"highlight")
;- _UIA_Action($oUIElement,"click")

 

[careca]

Thanks for the input, i've let the code run as you provided, and it eventually worked, with the item not visible, now the issue is that it seems to be doing the same thing as before, taking a long time to search or something. Seems like the highlight makes the hyperlink show up, and then it clicks, but takes a long time to get there, here's the output and log:

deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;RTI.MAINWINDOW
  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;RTI.MAINWINDOW
  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;RTI.SEARCHCONTEXT
  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;RTI.SEARCHCONTEXT

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171211-192529454"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="0" timestamp="20171211-192531381"> </logline>
<logline level="0" timestamp="20171211-192532201"> </logline>
<logline level="0" timestamp="20171211-192532884"> Fallback finding 1 object under Desktop    
_UIA_normalizeExpression ;elements 1-1in properties array
  _UIA_getObjectByFindAll property 1
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 11Found match with element: 0 the Name is: &lt;Title is: &lt;Close&gt;    Class   := &lt;Shell_LightDismissOverlay&gt;    controltype:= &lt;UIA_ButtonControlTypeId&gt;    ,&lt;50000&gt;    , (0000C350)    rect := &lt; 0;0;2560;2560&gt;    hwnd := &lt; 132792&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;Light Dismiss&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT Desktop 1
Storing in RTI as RTI.oP1.mainwindow
&lt;action&gt;Action 1 setfocus on oP1.mainwindow _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171211-192533097"> </logline>
<logline level="0" timestamp="20171211-192549147"> _UIA_action: Finding object oUIElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
  looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1&lt;propertymatching&gt;&lt;/propertymatching&gt;  walking back to mainwindow and deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
Desktop 1 parent0_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 6800Found match with element: 35 the Name is: &lt;Title is: &lt;More apps&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;724;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;OptionsButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 2 highlight on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171211-192549281"> </logline>
<logline level="0" timestamp="20171211-192605389"> _UIA_action: Finding object oUIElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
  looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1&lt;propertymatching&gt;&lt;/propertymatching&gt;  walking back to mainwindow and deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
Desktop 1 parent0_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 6800Found match with element: 35 the Name is: &lt;Title is: &lt;More apps&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;724;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;OptionsButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 3 setfocus on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171211-192605603"> </logline>
<logline level="0" timestamp="20171211-192605752"> _UIA_action: Finding object oUIElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
  looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1Found match with element: 0 the Name is: &lt;Title is: &lt;More apps&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;673;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;OptionsButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 4 highlight on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
<logline level="0" timestamp="20171211-192605882"> </logline>
<logline level="0" timestamp="20171211-192606019"> _UIA_action: Finding object oUIElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
  looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0&lt;propertymatching&gt;&lt;/propertymatching&gt;  deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
  _UIA_getObjectByFindAll property 1 Title:=More apps
 name:[Title] value:[More apps] having index 3
  _UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
 name:[controltype] value:[50005] having index 15
  _UIA_getObjectByFindAll property 3 class:=TouchHyperLink
 name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1Found match with element: 0 the Name is: &lt;Title is: &lt;More apps&gt;    Class   := &lt;TouchHyperLink&gt;    controltype:= &lt;UIA_HyperlinkControlTypeId&gt;    ,&lt;50005&gt;    , (0000C355)    rect := &lt; 1081;673;390;50&gt;    hwnd := &lt; 0&gt;    acceleratorkey := &lt; &gt;    automationid := &lt;OptionsButton&gt;

&lt;propertymatching&gt;&lt;/propertymatching&gt;Storing parent for found object in RTI as RTI.PARENT
&lt;action&gt;Action 5 click on oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0&lt;/action&gt;</logline>
</log>

 

These are the lines that take longer

_UIA_Action("oUIElement","highlight")
_UIA_Action("oUIElement","setfocus")

Once it gets to the highlight, the whole desktop gets a red mark, it highlights the whole desktop.

 

Edited December 11, 2017 by careca

[junkew]

Not sure where it goes wrong but this is not nice in the log  (in my system its 5600 so indeed findall is not allways efficient

walk thru the tree with n elements where n equals 6800
Found match with element: 35

So it loads first the 6800 elements in the arrray where it only needs number 35

suggestions

• The first oP1.mainwindow leave as is but the 2nd make it page2_P1.mainwindow
• Rewrite for the moment to this logic which makes sure you only retrieve one time the 6800 elements

$oUIElement=_UIA_Action("oUIElement","getobject")
_UIA_Action($oUIElement,"setfocus")
_UIA_Action($oUIElement,"highlight")
_UIA_Action($oUIElement,"click")

• Study the treewalker examples as that could be more efficient as then you only walk thru elements 1-34

The UIWrappers will not help you to do this much quicker without treewalker at the moment but however this thread has my interest to improve the uiawrappers ;-)

[junkew]

Ok here comes a full example that uses less object elements in searching

• Put in dutch and best educated guess of english in the regex descriptions
• Some tweaking on the hierarchy to come to element P10 in a quick way without the 1000's of objects of findall
• I just go to element 5 but you could also go down 1 by 1 to find .mp3
• After it fully runs check the log and see value of n (which should be low numbers) for finding:  

  walk thru the tree with n elements where n equals

• Specifically directing the element tree to follow in searching will decrease it as you only will search childs and not grandchildren

  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP2","getobject"))
  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP3","getobject"))
  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP6","getobject"))

   

Full demo on dutch windows version code (not cleaned but shows the power possibilities of UIA)

expandcollapse popup

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $iPID = ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
Sleep(5000) ;Give some time 10 seconds to start
consolewrite($ipid)

_UIA_setVar("oPage1.mainwindow","Title:=((Koppelingen instellen)|(Set Associations));ControlType:=UIA_WindowControlTypeId")  ;class:=CabinetWClass")    ;Koppelingen instellen
_UIA_setVar("oP2","Title:=((Koppelingen instellen)|(Set Associations));controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass")  ;Koppelingen instellen
_UIA_setVar("oP3","controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName")  ;
;~ _UIA_setVar("oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView") ;Deelvenster Verkenner
;~ _UIA_setVar("oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Deelvenster Mapindeling
_UIA_setVar("oP6","controltype:=UIA_PaneControlTypeId;class:=XElement") ;
;~ _UIA_setVar("oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby") ;
;~ _UIA_setVar("oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage")   ;Een bestandstype of protocol aan een specifiek programma koppelen
;~ _UIA_setVar("oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Een bestandstype of protocol aan een specifiek programma koppelen
_UIA_setVar("oP10","Title:=((Lijst met extensies)|(List with extensions));ControlType:=UIA_ListControlTypeId;class:=CCListViewEx")  ;Lijst met extensies
_UIA_setVar("oP11","Title:=((Extensies)|(Extensions));controltype:=UIA_GroupControlTypeId") ;class:=")  ;Extensies
;~ _UIA_setVar("oP12","Title:=.123;controltype:=UIA_ListItemControlTypeId;class:=") ;.123

;~ $oUIElement=_UIA_getObjectByFindAll(".123.mainwindow", "title:=.123;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=.123;controltype:=UIA_TextControlTypeId;class:=") ;ControlType:=UIA_TextControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles


_UIA_Action("oPage1.mainwindow","setfocus")
_UIA_Action("oPage1.mainwindow","highlight")
sleep(500)


;~ _UIA_Action("oP2","highlight")
;~ _UIA_Action("oP2","setfocus")
;~ sleep(500)
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP2","getobject"))
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP3","getobject"))
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP6","getobject"))
;~ _UIA_Action("oP3","setfocus")
sleep(500)
;~ _UIA_Action("oP4","highlight")
;~ _UIA_Action("oP4","setfocus")
;~ _UIA_Action("oP5","highlight")
;~ _UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
;~ _UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
;~ _UIA_Action("oP7","setfocus")
;~_UIA_Action("oP8","highlight")
;~ _UIA_Action("oP8","setfocus")
;~_UIA_Action("oP9","highlight")
;~ _UIA_Action("oP9","setfocus")

local $oListView=_UIA_Action("oP10","getobject")

_UIA_Action($oListView,"setfocus")
_UIA_Action($oListView,"highlight")
sleep(500)

_UIA_setVar("RTI.SEARCHCONTEXT", $oListView)

local $oListGroup=_UIA_Action("oP11","getobject")
_UIA_Action($oListGroup,"setfocus")
_UIA_Action($oListGroup,"highlight")
sleep(500)

;~ consolewrite(_UIA_getAllPropertyValues($oListView))

;~ _UIA_Action("oP11","highlight") ;commented as using getobject in uiaction
;~ _UIA_Action("oP11","setfocus")  ;commented as using getobject in uiaction
;~_UIA_Action("oP12","highlight")
;~ _UIA_Action("oP12","setfocus")

;~ _UIA_action("oUIElement","highlight")
;~ _UIA_action("oUIElement","setfocus")
;~ _UIA_action("oUIElement","click")


;~ Approach 1
;~ Go to the first in the list, not really needed
send("^{HOME}")

for $i=1 to 5
    send("{DOWN}")
    ;~ Check for the focused Element
    $UIA_oUIAutomation.GetFocusedElement($UIA_pUIElement )
    ;~ consolewrite("Element from point is passed, trying to convert to object ")
    local $oUIElement = objcreateinterface($UIA_pUIElement,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    consolewrite(_UIA_getAllPropertyValues($oUIElement))
Next

;~ Approach 2
;~ Determine some information on the listview
local $pElements
local $arrLength
$oListGroup.FindAll($treeScope_children, $UIA_oTRUECondition, $pElements)
local $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
;~  If there are ** NO ** childs found then there is nothing to search
If _UIA_IsElement($oAutomationElementArray) Then
;~ All elements to inspect are in this array assign the lenght of the array to variable
    $oAutomationElementArray.Length($arrLength)
    consolewrite($arrLength & " items found in the listview" & @CRLF)
Else
    consolewrite("FATAL: unexpected no childs found" & @CRLF)
    $arrLength = 0
EndIf

;~  For $i = 0 To $iLength - 1; it's zero based
for $i=0 to 5
    $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
    $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    local $tmpStr = "Title is: <" & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & ">" & @TAB _
                 & "Class   := <" & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & ">" & @TAB _
                 & "controltype:= " _
                 & "<" & _UIA_getControlName(_UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId)) & ">" & @TAB _
                 & ",<" & _UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId) & ">" & @TAB _
                 & ", (" & Hex(_UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId)) & ")" & @TAB _
                 & ", acceleratorkey:= <" & _UIA_getPropertyValue($oUIElement, $UIA_AcceleratorKeyPropertyId) & ">" & @TAB _
                 & ", automationid:= <" & _UIA_getPropertyValue($oUIElement, $UIA_AutomationIdPropertyId) & ">" & @TAB
    consolewrite($tmpStr & @CRLF)
Next

;And click the button
_UIA_Action("oPage1.mainwindow","setfocus")
_UIA_setVar("oButtonAssociate","title:=((Ander.*)|(Change program.*));classname:=CCPushButton")
;~ _UIA_Action("oButtonAssociate","highlight")
sleep(500)
_UIA_Action("oButtonAssociate","click")
sleep(500)

;Define the popup elements
_UIA_setVar("oP1Popup.MainWindow","Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")   ;Hoe wilt u .3gp-bestanden vanaf nu openen?
;~ _UIA_setVar("oP1","Title:=Hoe wilt u .3gp-bestanden vanaf nu openen?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")  ;Hoe wilt u .3gp-bestanden vanaf nu openen?
;~ _UIA_setVar("oP2","Title:=Flyoutvenster;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")    ;Flyoutvenster
;~ _UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=Element")   ;
;~ _UIA_setVar("oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Immersive Openwith Flyout
;~ _UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer") ;
;~ _UIA_setVar("oP6","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemListContainer")  ;
;~ _UIA_setVar("oP7","Title:=;controltype:=UIA_ListControlTypeId;class:=AppItemList")   ;
;~ _UIA_setVar("oP1Popup.oUIElement","Title:=Films.*;controltype:=UIA_ListItemControlTypeId;class:=AppListTileElement") ;ControlType:=UIA_ListItemControlTypeId;classname:=AppListTileElement")
_UIA_setVar("oP1Popup.oUIElement2","Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;ControlType:=UIA_HyperlinkControlTypeId;classname:=TouchHyperLink")

;Do the actions in the popup
_UIA_Action("oP1Popup.MainWindow","setfocus")
sleep(500)

;~ _UIA_Action("oP1Popup.oUIElement","highlight")
;~ sleep(500)

;~ _UIA_Action("oPopup.oUIElement","click")
sleep(500)


_UIA_Action("oP1Popup.oUIElement2","highlight")
_UIA_Action("oP1Popup.oUIElement2","setfocus")
sleep(500)
_UIA_Action("oP1Popup.oUIElement2","click")

 

• And for others reading above 
  yes, attaching extension(s) to a program can be done differently and not neccesary to do it thru GUI but its a nice example on what is possible
  Attaching best done thru registry manipulation or thru powershell script

[careca]

I thank you much for your help, i extracted the lines that relate to the flyout popup, and used my previous code to select the extension and click the button, so, as i said before, the  issue is really only on the popup itself.

This works, but it's still slow, can we focus only on this bit, and leave the rest?

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)
Local $sText1, $Vis
ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
            WinWaitActive("Set Associations", 'Set Associations', 10)
            Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
            If $hWnd <> 0 Or @error <> 0 Then
                Do
                    $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
                Until $Vis = 1
                Sleep(100)
                Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
                For $k = 0 To $iCount - 1
                    $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
                    If $sText1 = '.htm' Then
                        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
                        ConsoleWrite('=============================================================================' & @CRLF)
                        ConsoleWrite('Line - ' & $k & ' - ' & $sText1 & @CRLF)
                        ConsoleWrite('=============================================================================' & @CRLF)
                        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
                        Elem1()
                        ExitLoop
                    EndIf
                Next
            EndIf
;=============================================================================
Func Elem1()
;Define the popup elements
_UIA_setVar("oP1Popup.MainWindow","Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")
_UIA_setVar("oP1Popup.oUIElement2","Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")
;Do the actions in the popup
_UIA_Action("oP1Popup.oUIElement2","setfocus")
_UIA_Action("oP1Popup.oUIElement2","click")
EndFunc
;=============================================================================

Result is  

deep find in subtree Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;Desktop
deep find in subtree Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;RTI.SEARCHCONTEXT

Im sorry mate, im overwellmed trying to keep it to the popup portion only. any way to restrict the search on this?

You mentioned

_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP2","getobject"))
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP3","getobject"))
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP6","getobject"))

Can it be used in this instance?

;Define the popup elements
_UIA_setVar("oP1Popup.MainWindow","Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout") ;This is the popup
_UIA_setVar("oP1Popup.oUIElement2","Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;This is the "button" to search for within the popup aka the hyperlink "more apps"
;Do the actions in the popup
_UIA_Action("oP1Popup.oUIElement2","setfocus") ;Focus on the "button"
_UIA_Action("oP1Popup.oUIElement2","click") ;Click the button

Is that it? So why does it hang on the focus and the click?

BTW i found out that there's another parameter that maybe could be used to narrow the results.

AutomationID = OptionsButton

 

Edited December 12, 2017 by careca