IUIAutomation MS framework automate chrome, FF, IE, ....

[TheDcoder]

I am just glad that I found a workaround anyway. From the chrominum docs:

Quote

• MSAA/IAccessible (complete)
• IAccessible2 (mostly complete)
• ISimpleDOM (mostly complete)
• IAccessibleEx and UI Automation (very limited)

UI Automation support is very limited. So it makes sense why chrome has different results when using treewalkers and findall. Looks like I am sticking with my function

[InnI]

12 hours ago, TheDcoder said:

_UIA_getObjectByFindAll cannot find it

Because you find by $treescope_children (default). Try this

$oMuteButton = _UIA_getObjectByFindAll($oChrome, 'LegacyIAccessibleName:=Mute;ControlType:=UIA_ButtonControlTypeId', $treescope_subtree)

 

3 hours ago, junkew said:

When I try with youtube and simplespy the LegacyIAccessibleName is NOT coming for mute but it is filled for play button

If mouse cursor is over "mute" button, then LegacyIAccessible.Name of "mute" button is empty. But if you move mouse cursor from "mute" button, then LegacyIAccessible.Name will be "Mute".

[TheDcoder]

3 minutes ago, InnI said:

Because you find by $treescope_children (default).

It worked . Thanks for the advice, I did play around with the $treescope constants but I didn't try the subtree option... I must have been very tired, thanks again!

[Frenchy]

Hi, sorry for my english...

i am testing the ex04 with the calculator  so i have put the french name of different key in the prog ex04 (following the dump inside log)

 

so i have the error with the result (key =) .. following the dump, the key is "Résultat" but i have this error in log...

<logline level="5" timestamp="20171015-201912548"> Matching name transformed to propertynumber: 30005: for having value 1</logline>
<logline level="5" timestamp="20171015-201912564"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201912736"> Matching name transformed to propertynumber: 30005: for having value Additionner</logline>
<logline level="5" timestamp="20171015-201912765"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201912970"> Matching name transformed to propertynumber: 30005: for having value 3</logline>
<logline level="5" timestamp="20171015-201913011"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201913171"> Matching name transformed to propertynumber: 30005: for having value Résultat</logline>
<logline level="5" timestamp="20171015-201913174"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201913317"> UIA WARNING ** NOT ** found the pattern</logline>
</log>

so maybe the cause is the "é" ?

 

thanks for help

Terry

 

EDIT: problem resolved... in fact the name is not Résultat.... i have used UIspu.exe and i have had the right name.. "Est égal à"

Edited October 15, 2017 by Frenchy
problem relsoved

[junkew]

23 hours ago, Frenchy said:

Hi, sorry for my english...

i am testing the ex04 with the calculator  so i have put the french name of different key in the prog ex04 (following the dump inside log)

 

so i have the error with the result (key =) .. following the dump, the key is "Résultat" but i have this error in log...

<logline level="5" timestamp="20171015-201912548"> Matching name transformed to propertynumber: 30005: for having value 1</logline>
<logline level="5" timestamp="20171015-201912564"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201912736"> Matching name transformed to propertynumber: 30005: for having value Additionner</logline>
<logline level="5" timestamp="20171015-201912765"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201912970"> Matching name transformed to propertynumber: 30005: for having value 3</logline>
<logline level="5" timestamp="20171015-201913011"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201913171"> Matching name transformed to propertynumber: 30005: for having value Résultat</logline>
<logline level="5" timestamp="20171015-201913174"> UIA found the element</logline>
<logline level="5" timestamp="20171015-201913317"> UIA WARNING ** NOT ** found the pattern</logline>
</log>

so maybe the cause is the "é" ?

 

thanks for help

Terry

 

EDIT: problem resolved... in fact the name is not Résultat.... i have used UIspu.exe and i have had the right name.. "Est égal à"

Simplespy should give the values also. If not let me know as that would be ineresting then to understand.

[Danp2]

The latest version of SimpleSpy is crashing on this line under some circumstances

$oTW.getparentelement($oUIElement,$pTMPParentHandle)

See this thread for details.

 

[junkew]

You most likely have to be admin to use simplespy on that stuff. Initially I also had it breaking on W10 64 bits.

Adding below to simplespy.au3 fixes the issue

#RequireAdmin

[Danp2]

Yes, that does resolve the issue. Seems like there should still be a way to prevent the program from this hard crash.

[Chefito]

 

Hello, I have to use this udf in a windows xp. When I use control+w of the simplepy it gives me this error:

Quote

/TODO codetemplates.txt not available An error occurred when reading the file. & @CRLF"C:\Documents and Settings\Administrador\Mis documentos\UIA_V0\UIA_V0_51\simplespy.au3" (86) : ==> Variable must be of type "Object".:
$UIA_oUIAutomation.ElementFromPoint($tStruct,$UIA_pUIElement )
$UIA_oUIAutomation^ ERROR

I have framework .net 4.0 installed. I have not been able to install the program of link https://support.microsoft.com/es-es/help/971513/description-of-the-windows-automation-api . The link to xp is broken. How can I run ui automation on windows xp? I need to install something? Where can I download it?

Thank you very much.

[junkew]

XP is out of support so time to upgrade to W7 or W10. Its not part of .NET its part of the operating system as far as I know.
But you need UIAutomationCoreClient.DLL at least. I do not have XP so cannot help you.

[vkrisz81]

helloo guys!

so i needed something good and i found ff.au3.

it worked, but now its not. it cant find opened window, and because of it nothing else works.

WHY???

i mean when i start program with ffopenurl, same happens, cant find the given url as an opened window! WHY????????

[vkrisz81]

ok the standard connect example also does not work, most fresh firefox client. so i have nothing to access to my browsers.. nice work!

ie does not handle webpages, ff does not accessible from ff.au3.

any good news?:D

 

[vkrisz81]

as i can see the program standrardly cant find opened url tab, and it cant receive regedit data about firefox.. maybe newer versions are using different reg keys or smth like this, and ff.au3 cant access the program. mozrepl works, it can communicate, but nothing more..

so this also FAIL!

[Danp2]

There's a separate thread for FF.au3 so unsure why you are posting about it here. FWIW, it still works, but only on older versions of FF such as 54.x. The MozRepl add-on doesn't work with later versions of FF. Therefore, the FF.au3 UDF no longer works with the latest FF.

Since you are posting in the IUIAutomation thread, have you tried using this UDF for your automation needs?

[junkew]

This udf works perfectly with all recent browser versions including firefox.

[adwtoronto]

Hello,

A bit off topic, but I found this forum and trying to implement it in a script for Outlook O365 set-up but at a complete loss.  I need to read if the "Set up Outlook Mobile on my phone, too" is checked, if it is checked then un-check.

Here is the simplespy output...

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373
At least we have an element title: [Set up Outlook Mobile on my phone, too] class: []

Having the following values for all properties: 
Title is: <Set up Outlook Mobile on my phone, too>    Class   := <>    controltype:= <UIA_CheckBoxControlTypeId>    ,<50002>    , (0000C352)    800;331;239;19
*** Parent Information top down ***
3: Title is: <Welcome to Outlook>    Class   := <NUIDialog>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    520;46;800;468
"Title:=Welcome to Outlook;controltype:=UIA_WindowControlTypeId;class:=NUIDialog""    
2: Title is: <Welcome to Outlook>    Class   := <NetUIHWND>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    521;69;798;444
"Title:=Welcome to Outlook;controltype:=UIA_PaneControlTypeId;class:=NetUIHWND""    
1: Title is: <>    Class   := <>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    521;69;798;444
"Title:=;controltype:=UIA_PaneControlTypeId;class:=""    
0: Title is: <>    Class   := <>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    521;69;798;444
"Title:=;controltype:=UIA_CustomControlTypeId;class:=""    

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=Welcome to Outlook;controltype:=UIA_WindowControlTypeId;class:=NUIDialog")    ;Welcome to Outlook
_UIA_setVar("oP2","Title:=Welcome to Outlook;controltype:=UIA_PaneControlTypeId;class:=NetUIHWND")    ;Welcome to Outlook
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=")    ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_CustomControlTypeId;class:=")    ;

;~ $oUIElement=_UIA_getObjectByFindAll("SetupOutlookMobileonmyphone,too.mainwindow", "title:=Set up Outlook Mobile on my phone, too;ControlType:=UIA_CheckBoxControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Set up Outlook Mobile on my phone, too;controltype:=UIA_CheckBoxControlTypeId;class:=") ;ControlType:=UIA_CheckBoxControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Welcome to Outlook;controltype:=UIA_WindowControlTypeId;class:=NUIDialog", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Welcome to Outlook;controltype:=UIA_PaneControlTypeId;class:=NetUIHWND", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=", $treescope_children)    
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)    
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("SetupOutlookMobileonmyphone,too.mainwindow", "title:=Set up Outlook Mobile on my phone, too;ControlType:=UIA_CheckBoxControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Set up Outlook Mobile on my phone, too;ControlType:=UIA_CheckBoxControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

*** Detailed properties of the highlighted element ***
UIA_title:= <Set up Outlook Mobile on my phone, too>
UIA_text:= <Set up Outlook Mobile on my phone, too>
UIA_regexptitle:= <Set up Outlook Mobile on my phone, too>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <800;331;239;19>
UIA_ProcessId:= <10724>
UIA_ControlType:= <50002>
UIA_LocalizedControlType:= <check box>
UIA_Name:= <Set up Outlook Mobile on my phone, too>
UIA_HasKeyboardFocus:= <True>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <True>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <1>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Set up Outlook Mobile on my phone, too>
UIA_LegacyIAccessibleRole:= <44>
UIA_LegacyIAccessibleState:= <1048596>
UIA_LegacyIAccessibleDefaultAction:= <Uncheck>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:10724,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.DLL)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

I can right now just do the action to click on it..

#include "UIAWrappers.au3"

_UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)

$oP0=_UIA_action("title:=Set up Outlook Mobile on my phone, too;ControlType:=UIA_CheckBoxControlTypeId", "getobject")
_UIA_action($oP0,"click")
 

Please help and Thanks!!

 

 

[junkew]

Check the spy output with on and off and check which property changed. Most likely the togglestate property changes and then i assume you can also set it with the togglepattern.

[traff]

Hi there, 

 

first of all: Thank you for this awesome set of wrapper functions, it's awesome what you can do with it. I've been working with AutoIt for fun for quite a while now, and stumbled across this by accident. Keep up the good work!

Still, I found an issue within Func uia_action(...)

 Given:

While True
    $result = _UIA_action($myObj, "getValue")
    FileWrite($myFile, $result)
WEnd

$result will sometimes be empty, as it appears that the clipboard requires more time to handle the new information given by Send(^c), so ClipGet() will return "".

From UIAWrappers.au3:

Case "getValue"
            $obj2ActOn.setfocus()
            Send("^a")
            Send("^c")
            $retValue = ClipGet()

 

I solved this by adding a static sleep in there:

Case "getValue"
            $obj2ActOn.setfocus()
            Send("^a")
            Send("^c")
            Sleep(100)
            $retValue = ClipGet()

Still, if the clipboard is busy by whatsoever, this will not solve the issue. I am not sure if there is a more sophisticated way to handle this?

Thanks and Regards!

[junkew]

Probably integrating these functions can help further

https://www.autoitscript.com/autoit3/docs/libfunctions/_ClipBoard_Empty.htm

[Gaigai]

On ‎2016‎年‎11‎月‎11‎日 at 4:38 AM, junkew said:

On ‎2014‎年‎1‎月‎18‎日 at 9:12 PM, LarsJ said:

; Click (invoke) Save button       Local $pInvoke, $oInvoke       $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )       $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )       If Not IsObj( $oInvoke ) Then Return $S_OK       $oInvoke.Invoke()       ConsoleWrite( "Save button clicked" & @CRLF & @CRLF )

sometimes,the save is ok,but sometimes it has click the dropdown button and it has not save