IUIAutomation MS framework automate chrome, FF, IE, ....

LarsJ · November 20, 2017

It's the example in post 126.

I've just tried the example 10 times in a row with inspect.rar here. The file was saved automatically every time. You can try the same.

What files do not work? Are there any specific characteristics for the files or the web sites? Do you have any examples of files that failed, so I can try myself?

junkew · November 20, 2017

Alternative when invoke is not working to calculate the center of the element coordinates, do mousemove and mouseclick. See details in uiawrappers.au3

LarsJ · November 20, 2017

junkew, I think the problem here is that Invoke actually works. The Save button is clicked, the dialog box disappears, but the file is not saved anyway.

The standard solution is to insert a Sleep() eg. Sleep(200) immediately before the code section with the Save button click. This gives the download a little bit more time to finish before the Save button is clicked.

Edited November 21, 2017 by LarsJ
Last sentence

Jasp402 · November 28, 2017

I need help.
What do I need to do?
I need to be able to get the text of a table. I started testing with "ControlListView" but it returned "0". Was when I found "IUIAutomation MS framework automates Chrome, FF, IE, ...." I decided to try.

try with the inspect that I will follow @TheDcoder and it is much more complete. it extracts almost all the necessary information, but still I am still unable to access the text of the table.

I would like to know if these values represent any problem or if the contents of the table can still be accessed.

Grid.ColumnCount:       [Not supported]
Grid.RowCount:       [Not supported]
GridItem.Column:       [Not supported]
GridItem.ColumnSpan:   [Not supported]
GridItem.ContainingGrid:[Not supported]
GridItem.Row:           [Not supported]
GridItem.RowSpan:       [Not supported]

In the log.txt file there more info.

log.txt

Edited November 28, 2017 by Jasp402

junkew · November 28, 2017

I doubt if it works seems to be outdated grid solution

I googled the grid name

https://www.codeproject.com/Articles/20183/The-Ultimate-Grid-Home-Page

https://community.softwaregrp.com/t5/UFT-Practitioners-Forum/How-to-identify-third-party-custom-Dundas-control-using-QTP-9-5/td-p/394403

Jasp402 · November 28, 2017

31 minutes ago, junkew said:

I doubt if it works seems to be outdated grid solution

I googled the grid name

https://www.codeproject.com/Articles/20183/The-Ultimate-Grid-Home-Page

https://community.softwaregrp.com/t5/UFT-Practitioners-Forum/How-to-identify-third-party-custom-Dundas-control-using-QTP-9-5/td-p/394403

...I no longer feel very hopeful :rip:

NassauSky · November 29, 2017

Does anyone know of an issue with simplespy.au3 giving an error during running:

Line 4 (File "MSAccessibility.au3");

#include <autoitconstants.au3>

Error: Error opening the file.

Any idea?

Thanks.

Earthshine · November 29, 2017

your AutoIt installation may be corrupt. the file lives here by default. Check if it is there.

C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3

worst case you have to uninstall and reinstall latest AutoIt installation?

Edited November 29, 2017 by Earthshine

NassauSky · November 29, 2017

That was exactly it. Thanks @Earthshine

Earthshine · November 29, 2017

6 minutes ago, nassausky said:

That was exactly it. Thanks @Earthshine

you may want to schedule a disk check

at a command prompt type

Chkdsk c: /s

then reboot and let it fix up the drive. Sometimes files go missing when the OS hangs and it gets hard booted

NassauSky · November 29, 2017

Appreciate that tip. Good idea @Earthshine

Thanks will do.

Earthshine · November 29, 2017

sorry mate, it's

chkdsk /F

that fixes stuff

Edited November 29, 2017 by Earthshine

NassauSky · December 13, 2017

Hi @junkew

For anyone interested I figured out what my problem was with a newer version of chrome. Anyone who is an expert at this would have figured this out in a couple minutes.

I was getting an error running the ex5_chrome example.

D:\EXAMPLES_V0_64\ex5_chrome.au3" (116) : ==> Variable must be of type "Object".:
$oValueP.CurrentValue($myText)
$oValueP^ ERROR

Chrome is version Version 63.0.3239.84 (Official Build) (64-bit)

This is the dump log file. I've been looking at a couple examples all day trying to see how to traverse the elements using this threads technique. Wish there was an easier way.

<?xml version="1.0" encoding="UTF-8"?>
<log space="preserve">
<logline timestamp="20171212-193701870" level="5">
<information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information>
</logline>
<logline timestamp="20171212-193701870" level="5"> Matching class transformed to propertynumber: 30012: for having value Chrome_WidgetWin_1</logline>
<logline timestamp="20171212-193701932" level="5"> UIA found the element</logline>
<logline timestamp="20171212-193702120" level="0">
<treedump>
<treeheader>***** Dumping tree *****</treeheader>
<elementinfo>Title is: <> Class := <Intermediate D3D Window> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) , acceleratorkey:= <> , automationid:= <> </elementinfo>
<elementinfo>Title is: <> Class := <> controltype:= <UIA_TitleBarControlTypeId> ,<50037> , (0000C375) , acceleratorkey:= <> , automationid:= <> </elementinfo>
<elementinfo>Title is: <Browser container> Class := <> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) , acceleratorkey:= <> , automationid:= <> </elementinfo>
</treedump>
</logline>
</log>

Full day lesson. Figured out for the chrome toolbar the matchstring should be:

switch @OSLang
          case 0409 ;OS Language = en-US
              ;-$MatchString="start" ;- Doesn't work form Chrome v63.0.3239.84
              $MatchString= "controltype:=" & $UIA_CustomControlTypeId ;

For the chrome address bar

local $oChromeAddressBar=_UIA_getFirstObjectOfElement($oChromeToolbar,"controltype:=" & $UIA_EditControlTypeId , $treescope_subtree) ;works in chrome 29

So far I think I'm liking this automation but oh man it's way harder than ie.au3. I didn't even get to the part of traversing the DOM to find html elements. I need more lessons :-O

Edited December 13, 2017 by nassausky
Typo

junkew · December 13, 2017

Its much more powerfull then ie.au3 and in that sense maybe more complicated but just let me know what you find easier on ie.au3 so I can improve the uiawrappers

Chhrome examples I will fix with next update.
Variable must be of type object I will make library more robust but indeed in the end means incorrect identification properties for object.

Some tips

Understand your UI Tree with the different spying tools around. I am using
inspect.exe but sometimes freezes,
VisualUIAVerify.exe cannot see them all (especially popups that dissapear when you want to inspect),
Simplespy made by myself to tweak more into what I need in my day to day job (and I frequently add some small parts to identify specific scenario's that are hard to generalize)
Understand visible and non visible grouping controls in the parent hierarchy
Understand the difference between finding in children and grandchildren (number of objects searching through)

u547899 · January 10, 2018

Is there some way to speed up _UIA_getObjectByFindAll()? See gif...it's taking nearly 2 seconds to identify each element

I'm running the following code in a loop that identifies each of the checkboxes. The $subsection variable is as close as I can get to the direct parent of those elements when using simplespy.

$element = _UIA_getObjectByFindAll($subsection, 'title:=' & $consultation_of.Keys[$i] & ';classname:=WindowsForms10.BUTTON.app.0.202c666', $treescope_subtree)

Here's the simplespy output for the "Shoulder" checkbox:

   https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 
At least we have an element title: [Shoulder] class: [WindowsForms10.BUTTON.app.0.202c666]

Having the following values for all properties: 
Title is: <Shoulder>    Class   := <WindowsForms10.BUTTON.app.0.202c666>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    1420;531;86;17
*** Parent Information top down ***
14: Title is: <Note Authoring>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    951;72;935;936
"Title:=Note Authoring;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
13: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    955;119;927;885
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
12: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    955;119;927;885
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
11: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    955;119;927;885
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
10: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    955;178;927;826
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
9: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    955;178;927;826
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
8: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_TabControlTypeId>    ,<50018>    , (0000C362)    965;188;907;806
"Title:=;controltype:=UIA_TabControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
7: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    966;209;905;784
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
6: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    966;209;905;784
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
5: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    966;209;905;784
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
4: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1168;241;703;752
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
3: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1168;274;703;585
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
2: Title is: <twNoteInputTemplate1>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1168;274;703;585
"Title:=twNoteInputTemplate1;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
1: Title is: <History of Present Illness>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1180;286;663;3007
"Title:=History of Present Illness;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    
0: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.202c666>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1190;312;643;2981
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=Note Authoring;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;Note Authoring
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP6","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP7","Title:=;controltype:=UIA_TabControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP8","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP9","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP10","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP11","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP12","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;
_UIA_setVar("oP13","Title:=twNoteInputTemplate1;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;twNoteInputTemplate1
_UIA_setVar("oP14","Title:=History of Present Illness;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;History of Present Illness
_UIA_setVar("oP15","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666")    ;

_UIA_setVar("Shoulder.mainwindow","title:=Shoulder;classname:=WindowsForms10.BUTTON.app.0.202c666")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
_UIA_Action("oP7","setfocus")
;~_UIA_Action("oP8","highlight")
_UIA_Action("oP8","setfocus")
;~_UIA_Action("oP9","highlight")
_UIA_Action("oP9","setfocus")
;~_UIA_Action("oP10","highlight")
_UIA_Action("oP10","setfocus")
;~_UIA_Action("oP11","highlight")
_UIA_Action("oP11","setfocus")
;~_UIA_Action("oP12","highlight")
_UIA_Action("oP12","setfocus")
;~_UIA_Action("oP13","highlight")
_UIA_Action("oP13","setfocus")
;~_UIA_Action("oP14","highlight")
_UIA_Action("oP14","setfocus")
;~_UIA_Action("oP15","highlight")
_UIA_Action("oP15","setfocus")

_UIA_action("Shoulder.mainwindow","setfocus")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP14=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Note Authoring;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP14,"setfocus")
Local $oP13=_UIA_getObjectByFindAll($oP14, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP13,"setfocus")
Local $oP12=_UIA_getObjectByFindAll($oP13, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP12,"setfocus")
Local $oP11=_UIA_getObjectByFindAll($oP12, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP11,"setfocus")
Local $oP10=_UIA_getObjectByFindAll($oP11, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP10,"setfocus")
Local $oP9=_UIA_getObjectByFindAll($oP10, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP9,"setfocus")
Local $oP8=_UIA_getObjectByFindAll($oP9, "Title:=;controltype:=UIA_TabControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP8,"setfocus")
Local $oP7=_UIA_getObjectByFindAll($oP8, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP7,"setfocus")
Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=twNoteInputTemplate1;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=History of Present Illness;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.202c666", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar("Shoulder.mainwindow","title:=Shoulder;classname:=WindowsForms10.BUTTON.app.0.202c666")
_UIA_action("Shoulder.mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_title:= <Shoulder>
UIA_text:= <Shoulder>
UIA_regexptitle:= <Shoulder>
UIA_class:= <WindowsForms10.BUTTON.app.0.202c666>
UIA_regexpclass:= <WindowsForms10.BUTTON.app.0.202c666>
UIA_iaccessiblechildId:= <0>
UIA_id:= <TextFinding361>
UIA_handle:= <788282>
UIA_RuntimeId:= <42;788282>
UIA_BoundingRectangle:= <1420;531;86;17>
UIA_ProcessId:= <3392>
UIA_ControlType:= <50000>
UIA_LocalizedControlType:= <button>
UIA_Name:= <Shoulder>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <TextFinding361>
UIA_ClassName:= <WindowsForms10.BUTTON.app.0.202c666>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <788282>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <WinForm>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Shoulder>
UIA_LegacyIAccessibleRole:= <43>
UIA_LegacyIAccessibleState:= <1048576>
UIA_LegacyIAccessibleDefaultAction:= <Press>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:21272,hwnd:0xC073A Main:Nested [pid:3392,hwnd:0xC073A Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

BatMan22 · January 10, 2018

So quick question.. SimpleSpy works perfectly and is able to click what I couldn't 'see' before with window info tool.. And using the 'click' command works great. But... Is there any way to speed it up? Is it getting stuck somewhere or is that just how long the deep search function takes? I even changed the "UIAWrappers.au3" file to where the sleep function was 20ms and it didn't help.. I only have these three lines in my code, but it still takes 12.54seconds to complete on an i7 processor.. is something wrong (ie is it getting stuck somewhere) or is that realistic? Is there anything I can do to speed it up?

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_action("Title:=Analytical;controltype:=UIA_RadioButtonControlTypeId;class:=","click")

LarsJ · January 11, 2018

The purpose of the wrapper functions is first and foremost to make UI Automation as easy as possible, avoiding advanced functions like ObjCreateInterface. This function seems to be an insurmountable barrier for 99.99% of all AutoIt users. Not only in connection with UI Automation but generally.

The wrapper functions contains a lot of code to make it easy and to implement as much functionality as possible. As usual you'll get nothing for free. The cost here is especially the speed.

Once you've got the wrapper functions to work, you can optimize the speed by using the UI Automation objects and methods directly (without using the wrapper functions). See some of my examples for details.

The example above in post #617 can probably be optimized to about 200 milliseconds.

junkew · January 11, 2018

@u547899 @BatMan22 yes, the uia of ms in itself is very fast the wrappers indeed sacrifice some speed but as most teams are coming from manual testing that takes hours or days to execute they are very happy as its now blazingly fast in minutes or seconds running during lunch or overnight. But as said 99.99% of the people are a little afraid to use the objcreateinterface logic.

Basically with 1 call of findall and filtercondition on type checkbox you can retrieve all checkboxes and then you have directly a array with reference to all of them. These examples are in examples.zip.

The wrappers do this

Get all childs unfiltered from parent
Loop thru the returned array and filter with regular expression logic and capability to jump n elements back or forward which is not possible with uia interfaces out of the box

So above 2 steps happens for every uiaction wrapper function which slows down especially if you know that all checkboxes are already in the array.

BatMan22 · January 11, 2018

2 hours ago, junkew said:

@u547899 @BatMan22 yes, the uia of ms in itself is very fast the wrappers indeed sacrifice some speed but as most teams are coming from manual testing that takes hours or days to execute they are very happy as its now blazingly fast in minutes or seconds running during lunch or overnight. But as said 99.99% of the people are a little afraid to use the objcreateinterface logic.

Basically with 1 call of findall and filtercondition on type checkbox you can retrieve all checkboxes and then you have directly a array with reference to all of them. These examples are in examples.zip.

The wrappers do this

Get all childs unfiltered from parent

Loop thru the returned array and filter with regular expression logic and capability to jump n elements back or forward which is not possible with uia interfaces out of the box

So above 2 steps happens for every uiaction wrapper function which slows down especially if you know that all checkboxes are already in the array.

Thanks dude.. I'll try and figure out how all this works >.<, either way I appreciate your work, it really helps for idiots like me

junkew · January 12, 2018

Study post 3 example1 thats the raw interface with treewalkers and base condition filters.

This is the uiabase

Create a filter condition.
Search in examples for CreatePropertyCondition or in this thread
A true condition is the simplest (will not filter/give back all elements)
see https://msdn.microsoft.com/en-us/library/ms752331(v=vs.85).aspx#Using_Conditions
Do a findall with treescope, filter condition and the reference for the result
As said then it will only find on full matching and not on partial (which you frequently need if you deal with titles, multi language etc)

$UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

; Condition to find View menu item
  Local $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "View", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return

Thoughts for UIAwrappers (will put it on TODO list)

Keep the initial elements in array found of a certain element in memory and scan that first as most likely you will handle n controls under same parent in sequence
so UIA_SETVAR("RTI.SEARCHCONTEXT", $window) should set the window and the initial array and
UIA_Action("checkbox1","click"), UIA_Action("checkbox2","click") should reuse the array thats set as a searching context

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

junkew

junkew

junkew

Posted Images

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Similar Content