IUIAutomation learning

[caramen]

Hello peoples

 

Learning UIA. To make imba reliable scripts  

 

well i am getting it working in my learning process hihi but hmmmmmmmmmmmm i got an easy reproducer here that i guess you can use if you're including UIWrapper.au3 and dont forget to change $Window =  "tittle" 

 

Simple question how to change my exit code 17 sec to 0.3 seconde ? Some explanation would be welcome

 

 

#include "UIAWrappers.au3"

ShellExecute ("Appwiz.cpl")

$Window  = WinWaitActive ("Programmes et fonctionnalités")

If WinExists ($Window) Then

_UIA_setVar("oUIElement","Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox") ;ControlType:=UIA_EditControlTypeId;classname:=SearchEditBox")


_UIA_action("oUIElement","highlight")
_UIA_action("oUIElement","settextvalue" , "test settextvalue")
_UIA_action("oUIElement","setvalue" , "test setvalue")

EndIf

 

SCITE Console : 

>"C:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "C:\Users\Caramen\OneDrive\Autoit\Projet\reproducer string.au3"    
  deep find in subtree 1 oUIElement;RTI.MAINWINDOW
  deep find in subtree 2 Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox;RTI.MAINWINDOW
    RTI.MAINWINDOW is an object1
  deep find in subtree 1 oUIElement;RTI.MAINWINDOW
  deep find in subtree 2 Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox;RTI.MAINWINDOW
    RTI.MAINWINDOW is an object1
  deep find in subtree 1 oUIElement;RTI.MAINWINDOW
  deep find in subtree 2 Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox;RTI.MAINWINDOW
    RTI.MAINWINDOW is an object1
>Exit code: 0    Time: 17.96

 

Question 2 : How to disable the LOG too ? 

 

Just noticed that in @junkew's TO DO list :

• If speed becomes an issue use the caching logic of the MS UIA framework

So hmmm the speed of UIA is normal when i have 17 sec ? but that also mean there is a second solution to catch control ? 

 

Edit again :  

2 set value & 1 click  

>Exit code: 0    Time: 67.25

 

Please help me 

 

 

Yeaaaah improvement ! 1 set value & 2 clicks !!!!  

>Exit code: 0    Time: 12.32

 

Edited October 12, 2018 by caramen

[careca]

I got the same problem a while back, decided i was not going to bother with it anymore.

[junkew]

• UIAutomation in itself out of the box from Microsoft is in general fast.
• If you give the proper identification string it will work fast with the wrappers
• If you do not give a proper identification a "smart" uia wrapper algorithm tries to find your element anyway.
  So its a choice between slow finding your object of interest or failing directly
  If you have bad luck it will fully search your full desktop and all (sub)windows which will offcourse be terrible when html elements are around
• When you see deep find in your log its probably smart to have additional setfocus actions and/or look at your expression for identifying
• Ideas for smarter fallback algorithm are allways welcome
  rti.mainwindow and rti.parent are important parts for the algoritm in fallback searching

In general the hierarchy of your object you can see with simplespy

• Desktop
  • Window 1
  • Window 2
    • Group element
      • your element A
    • Group element
      • Group element 
        • Group element B

If you directly try:

_UIA_Action("Title:=B","click")

If will start searching from desktop and indirectly hit all open windows and its childs so in above when window 1 is a browser its traversing all html elements before it goes to window 2.
 

Below would probably already speedup as you first put focus to mainwindow with a UIA function (mixing with AutoIt standard funtions is in general not a good way to start)

#include "UIAWrappers.au3"

ShellExecute ("Appwiz.cpl")

$Window  = WinWaitActive ("Programmes et fonctionnalités")

If WinExists ($Window) Then

_UIA_setVar("my program.mainwindow","Title:="Programmes et fonctionnalités")
_UIA_setVar("oUIElement","Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox") ;ControlType:=UIA_EditControlTypeId;classname:=SearchEditBox")

_UIA_action("my program.mainwindow","highlight")
_UIA_action("my program.mainwindow","setfocus")

_UIA_action("oUIElement","highlight")
_UIA_action("oUIElement","settextvalue" , "test settextvalue")
_UIA_action("oUIElement","setvalue" , "test setvalue")

Edited October 13, 2018 by junkew

[careca]

10 hours ago, junkew said:

• If you give the proper identification string it will work fast with the wrappers

 

I think this is where i have most trouble, i seem to never find a proper identification string, and i dont know what it looks like, can you paste an example?

In the past, it would seem that as many elements for identification of a control i had, the system would always fall back to searching all over.

[junkew]

There are dozens of examples in the zip of uiautomation.

What example are you missing.

1. Use id or title will work best
2. Use index, indexrelative
3. More complex examples with 117 properties you can combine in regular expressions.
4. Simplespy will give you a starting point for descriptions

Read and study the examples. There are many descriptions there automating the forum with all major browsers

 

Edited October 14, 2018 by junkew

[caramen]

I did that 

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP0=_UIA_getObjectByFindAll($UIA_oDesktop, "title:=Zone de recherche;ControlType:=UIA_EditControlTypeId", $treescope_children)
_UIA_Action($oP0,"setfocus")
_UIA_setVar("Programmes et fonctionnalités.mainwindow", "Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox")
_UIA_action("Programmes et fonctionnalités.mainwindow","setvalue" , "test setvalue")

But now it s no more working what i am missing ?

SimpleSpy : 

expandcollapse popup

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 
At least we have an element title: [Zone de recherche] class: [SearchEditBox]

Having the following values for all properties: 
Title is: <Zone de recherche>   Class   := <SearchEditBox>  controltype:= <UIA_EditControlTypeId>   ,<50004>    , (0000C354)    595;137;247;17
*** Parent Information top down ***
6: Title is: <Programmes et fonctionnalités>   Class   := <CabinetWClass>  controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    82;99;800;505
"Title:=Programmes et fonctionnalités;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass""  
5: Title is: <> Class   := <WorkerW>    controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    89;129;785;34
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WorkerW""    
4: Title is: <> Class   := <ReBarWindow32>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    89;129;785;34
"Title:=;controltype:=UIA_PaneControlTypeId;class:=ReBarWindow32""  
3: Title is: <> Class   := <UniversalSearchBand>    controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    590;132;277;27
"Title:=;controltype:=UIA_PaneControlTypeId;class:=UniversalSearchBand""    
2: Title is: <> Class   := <Search Box> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    590;134;277;24
"Title:=;controltype:=UIA_PaneControlTypeId;class:=Search Box"" 
1: Title is: <> Class   := <SearchEditBoxWrapperClass>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    593;134;271;24
"Title:=;controltype:=UIA_PaneControlTypeId;class:=SearchEditBoxWrapperClass""  
0: Title is: < Rechercher dans : Programmes et fonctionnalités>    Class   := <SearchBox>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    593;134;271;24
"Title:= Rechercher dans : Programmes et fonctionnalités;controltype:=UIA_PaneControlTypeId;class:=SearchBox"" 


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=Programmes et fonctionnalités;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass")    ;Programmes et fonctionnalités
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WorkerW")  ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=ReBarWindow32")    ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_PaneControlTypeId;class:=UniversalSearchBand")  ;
_UIA_setVar("oP5","Title:=;controltype:=UIA_PaneControlTypeId;class:=Search Box")   ;
_UIA_setVar("oP6","Title:=;controltype:=UIA_PaneControlTypeId;class:=SearchEditBoxWrapperClass")    ;
_UIA_setVar("oP7","Title:= Rechercher dans : Programmes et fonctionnalités;controltype:=UIA_PaneControlTypeId;class:=SearchBox")   ; Rechercher dans : Programmes et fonctionnalités

;~ $oUIElement=_UIA_getObjectByFindAll("Zonederecherche.mainwindow", "title:=Zone de recherche;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox") ;ControlType:=UIA_EditControlTypeId;classname:=SearchEditBox")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight"

 

Edited October 15, 2018 by caramen

[caramen]

Starting to understand and getting it work fast. 

 

Well urgh it s not very simple xD. 

 

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)


Local $oP0=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Programmes et fonctionnalités;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass", $treescope_children)

_UIA_setVar("oP0","Title:=Programmes et fonctionnalités;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass")
_UIA_Action($oP0,"setfocus")

local $oUIElement=_UIA_getObjectByFindAll("Zonederecherche.mainwindow", "title:=Zone de recherche;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Zone de recherche;controltype:=UIA_EditControlTypeId;class:=SearchEditBox") ;ControlType:=UIA_EditControlTypeId;classname:=SearchEditBox")

;~ _UIA_action($oUIElement,"highlight")
_UIA_action("oUIElement","setvalue" , "test setvalue")

 

>Exit code: 0    Time: 2.949

 

Now my last question... I noticed all click are like a simple mouseclick and are not send like controlsend no way to send in minimised things ? 

Edited October 15, 2018 by caramen

[mLipok]

1 minute ago, caramen said:

and getting it work fast. 

any stats ?

 

[caramen]

4 minutes ago, caramen said:

>Exit code: 0    Time: 2.949

Y i edited sorry

 

Now my last question... I noticed all click are like a simple mouseclick and are not send like controlsend no way to send in minimised things ? 

Edited October 15, 2018 by caramen

[mLipok]

22 minutes ago, caramen said:

send in minimised things

? 

I'm not sure if you are english native speakers, but I'm not.
Could you use other wording/description ?

ps.
This is intereting topic for me so I wanted to be sure I understand what you think .... to be able fully understand @junkew answer

 

[caramen]

Junkew is not the only one to be able to unswer,   

 

I want to say : 

When you use controlclick it can be send when the software is in minimized stat. 

 

If you use a click from the UIAutomation UDF it wont be sent as a controlclick. it will be like a normal mouseclick. And the mouse click can click only on visible elements.

 

The common use is to script let the script do stuff and you can do stuff at same time without interacting with script. 

Edited October 15, 2018 by caramen

[mLipok]

27 minutes ago, caramen said:

Junkew is not the only one to be able to unswer,  

I know but I thought that he is already involved into answering in your topic, so I assume that he will be the first who answer

 

[junkew]

Click by mouse is most close to enduser simulation and click on x y position is most general when you determine your x y h w of any gui element.

Controlclick either relies on sendmessage or postmessage which in theory can bypass disabled buttons.

Within IE a direct call to method click of javascript can for same reasons bypass disabled controls.

Endusers will click on visible controls.

In general if you just want to automate a task you should not do it thru gui but use api thats frequently around to do things easier.

$p0 is something else then "p0" please double check if you understand the difference.

Its not difficult. Understand

(Grand)parent hierarchy and its just highlevel

Desktop appwindow setfocus /

Grandparent setfocus

Parent setfocus

Element click

 

[caramen]

@junkew I think...I just got it. To make it the fastest way possible. I have to practice about the hierarchy of my windows ?

 

Desktop    ;SetFocus 

• Window 1
• Window 2 ;SetFocus2
  • Group element ;SetFocus3
    • The element i want A ;ElementClick

 

Edited October 15, 2018 by caramen

[junkew]

Yes, only desktop setfocus is normally not needed as you normally know which window is your mainwindow. Problem is that in general you encounter a deeper hierarchy of grouping windows which are there but not allways visible. Some windows match your description but are hidden so you cannot setfocus thats why you need inspect.exe to see other parts of gui hierarchy and simplespy to quickly tell the parents. For browsers its then hard to have one way of finding addresbar as all browsers have different hierarchy.

[caramen]

Yeah. k Thx.

WinList show a lot of invisible windows.