IUIAutomation MS framework automate chrome, FF, IE, ....

[junkew]

If its hanging 99.99% sure you do not have clear your UI hierarchy and you do not have a proper identificationstring

you mix au3 functions control* with _UIA* functions.
Thats in general a bad approach which will lead to issues.

• To learn this udf I advice to start in post 1 and with example 1 and make it work on calculator and notepad.
• Understand the first 10 examples and then start on your own application.
• Simplespy will help you by giving code as a present which you can continue to build with.
   

First understand the base and use highlight action before doing any other actions

#include "UIAWrappers.au3" 

_UIA_SETVAR("Description","classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad118")

_UIA_Action("Description","highlight")

;~_UIA_Action("Description","focus")

And read the first few posts in detail otherwise I am repeating what is already in start of this thread

[milkmoron]

 

tTrying the first example but i get Error: Unknown Fuction Name.  I can't find these functions in autoit help either. What do these mean?

samplewinlist()

sampleTW(1)
sampleTW(2)
sampleTW(3)

Nvm found the function in the example file.

 

I am using simplespy to define the vars but It seems that the program is getting the info for a different part of the window. Is there another way to fetch the info to define the var?

I'm notiding it says Enteredby: as the title but that i beleive is a different part of the window. The part im trying to select is the Description and that is not being highlighted when i select Description: or Description as the title

_UIA_setVar("oP1","Title:=View Activity;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")  ;View Activity
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1") ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1") ;
_UIA_setVar("oP4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")   ;Delete

_UIA_setVar("EnteredBy:.mainwindow","title:=Entered By:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

This is the whole code.

expandcollapse popup

AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=View Activity;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;View Activity
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oP4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Delete

_UIA_setVar("Description:.mainwindoww","title:=Entered By:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")

;~_UIA_Action("Description:","highlight")
_UIA_Action("Description:.mainwindow","setfocus")

Local $oDescription = _UIA_Action("Description:.mainwindow","getvalue")

_UIA_setVar("oFIADescription1","Title:=Activity / Suspense;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Activity / Suspense
_UIA_setVar("oFIADescription2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oFIADescription3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;
_UIA_setVar("oFIADescription4","Title:=Delete;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Delete
_UIA_setVar("oFIADescription5","Title:=Company:;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.30495d1_r9_ad1")    ;Company:
_UIA_setVar("oFIADescription6","Title:=Activity;controltype:=UIA_GroupControlTypeId;class:=WindowsForms10.BUTTON.app.0.30495d1_r9_ad1")    ;Activity

_UIA_setVar("FIADescription:.mainwindow","title:=Description:;classname:=WindowsForms10.EDIT.app.0.30495d1_r9_ad1")

;~_UIA_Action("oFIADescription1","highlight")
_UIA_Action("oFIADescription1","setfocus")
;~_UIA_Action("oFIADescription2","highlight")
_UIA_Action("oFIADescription2","setfocus")
;~_UIA_Action("oFIADescription3","highlight")
_UIA_Action("oFIADescription3","setfocus")
;~_UIA_Action("oFIADescription4","highlight")
_UIA_Action("oFIADescription4","setfocus")
;~_UIA_Action("oFIADescription5","highlight")
_UIA_Action("oFIADescription5","setfocus")
;~_UIA_Action("oFIADescription6","highlight")
_UIA_Action("oFIADescription6","setfocus")

;~_UIA_Action("FIADescription:.mainwindow","highlight")
_UIA_Action("FIADescription:.mainwindow","setfocus")
_UIA_Action("FIADescription:.mainwindow","settextvalue",$oDescription)

 

Edited July 20, 2018 by milkmoron

[junkew]

Use inspect.exe and see faq31 on tools that will tell you the hierarchy of your elements. You will have to read a lot and try things out. Its explained really in the beginning and for starters first advice is to automate calculator, notepad of windows. As you already see you think its a different element for that to know its true only inspect.exe can tell you. For sure nobody can tell based on what you post. We do not have access to your application so we can only guess what the actual hierarchy is.

[milkmoron]

Is there a list of functions to use with IUI. Maybe a description too?

[TheDcoder]

@milkmoron Yes, just open the included wrapper script and and look at the functions . Pro Tip: If you have full SciTE4AutoIt3 installed then you can use Alt + Q to open SciTE Jump which lists all of the functions!

[junkew]

@milkmoron please tell me why first post is not enough on explaining the functions and actions available then i can improve first post. And as given you can allways look in the udf module itself.

[milkmoron]

23 hours ago, junkew said:

@milkmoron please tell me why first post is not enough on explaining the functions and actions available then i can improve first post. And as given you can allways look in the udf module itself.

we'll it's a little hard to know what functions are available but I guess looking into the wrapper itself helps alot. A list of functions and a brief description of what they do would be nice. Like the Autoit help file.

[toto22]

I can't figure out how to use partial Window-Title. I need it because it constantly changes.

So this works 

Local $oP4 = _UIA_getObjectByFindAll($UIA_oDesktop,"Title:=Untitled - Notepad")

However, this doesn't

Opt("WinTitleMatchMode",2)
WinActivate("Note")
Local $sText = WinGetTitle("[ACTIVE]")
Local $sText2 = '"Title:=' & $sText & '"'
Local $oP4 = _UIA_getObjectByFindAll($UIA_oDesktop,$sText2)

Above are just examples. I'm trying to get it to work for Trade-Ideas program that has multiple windows of same class with constantly changing titles

[junkew]

Either use regex like 

Untitled.*

Or use

class:=..... with index:=...

 

[Hisma]

Hello,

I'm trying to figure out how to open a menu item in a program that has a non-traditional menu structure.  

I had the script working in an old version of the software like this - 

WinMenuSelectItem($handle, "", "&FILE", "Save As...")

Which obviously did a simple file -> save as.  

But the program has since been updated by the vendor, and now menu is custom-built.  Using the simply spy tool, when I highlight the "save as" menu item and press CTRL + W, I get this output (standard code flexible) - 

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Logix Designer - HP_RCS_VSC in HP_VSC.ACD [1756-L82ES 31.11];controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010003:00000000:019F1155", $treescope_children) 
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=File;controltype:=UIA_MenuControlTypeId;class:=Afx:00400000:800:00010003:00000010:00000000", $treescope_children)  
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=FILE;controltype:=UIA_ToolBarControlTypeId;class:=BCGPToolBar:400000:8:10003:10", $treescope_children) 
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("SaveAs....mainwindow", "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

As you can see, it's qutie large, and I can't figure out how to clean up that output to make it usable in my script.  Can someone help me please?

[junkew]

Is the code working? In general you have to follow gui hierarchy and first set focus to parent before invoking new childsearch.

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Logix Designer - HP_RCS_VSC in HP_VSC.ACD [1756-L82ES 31.11];controltype:=UIA_WindowControlTypeId;class:=Afx:00400000:8:00010003:00000000:019F1155", $treescope_children) 
_UIA_Action($oP2,"setfocus")

;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("SaveAs....mainwindow", "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP2, "title:=Save As...;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

Sometimes you can leave out some elements like above only topwindow p2 and search in subtree of p2.

[lundbird]

Hi Junkew. I like your library. I took a look at it and used similar ideas to make a GUI automation package in python. It works pretty much the same as how your code works but its a bit faster and available on PIP. Anyone can try it out, and I would love feedback. I also am using an expansion of the recorder-spy in the package.

I made a getting started guide on medium for those interested.

https://medium.com/@lundbird/how-to-automate-windows-applications-with-autogui-626c7b452eed 

[mLipok]

On 11.08.2013 at 12:04 AM, junkew said:

UIA_V0_66.zip                 EXAMPLES_V0_66.zip

EXAMPLES_V0_66.zip

UIA_V0_66.zip

First link to examples have wrong title.
Could you edit OP ?

 

[junkew]

done. but unsure what caused the 2 times showing of latest version

[mLipok]

I just start to check your examples, a little..... and in first place I change the names to have better ordering in sorting by name.
I change only file names, but hope this will help others, and I hope you will find this as a good contribution

 

 

Examples.zip

Edited October 10, 2018 by mLipok

[Earthshine]

I've been using this kit to help me test my C# toolset for perusing objects. I can test it without a real dev env using just autoit and .net... lol... so cool. I can find and enum objects super fast.

[Zein]

I've been using this to automate C# screens for a while, so I'm very thankful for all your work junkew. 

My question is that there seem to be some sort of inconsistency. I'm not sure if I'm using a more outdated approach, but the way I target C# objects is by first identifying the window I want to click in, and then making recursive functional calls to iterate through the tree of elements using a RawViewWalker to search for the element which matches the automation ID I'm targeting. 

I then have an order of operations that clicks various buttons and boxes on the screen. But sometimes the RawViewWalker doesn't seem to be able to find the elements I want to click despite them very clearly being there. Is there a known reason this could be the case? I get the vibe that it's because the screen is partially dynamic and some elements are added/updated after a click, and the walker is using a cached version of the window instead of the updated on.
 

[junkew]

You can only check with inspect.exe but thats probably a fresh treecache in a different process.

maybe MS documented it somewhere

https://docs.microsoft.com/en-us/dotnet/framework/ui-automation/caching-in-ui-automation-clients

 

if you know the automation id then probably findone or findfirst method will help you to find it with a filter condition on automationid instead of walking around the tree yourself

 

[Zein]

Thanks, where can I find the documentation/usage of findall/findone? Is there a wiki page for the entire framework anywhere?

Did a search/scan of the thread and no results. Tried to find info on the wiki and couldn't either.

[junkew]

hmmmm first post 2nd line