Jump to content

IUIAutomation MS framework automate chrome, FF, IE, ....


junkew
 Share

Recommended Posts

This is not my code you are running. Have you tried just to run my code as it is in Scite editor. I am pretty sure my code is working.

Before you run the code you must open HOBOnode Viewer Utility, select the "Data Search" tab, and make sure the "Save Data..." or "Get/Refresh Device List" button is enabled.

I can see, that your code fails on the "Find" section. To figure out what's wrong, I need to see the complete code.

Link to comment
Share on other sites

Sorry about that LarsJ. Yeah the previous error code came out from my code. If I run only your code I get this error. I'm glad that you are able to have it working. That gives me hope. I know I am pretty close on this. 

>"C:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "C:\Users\richi\Desktop\HOBO_MAIN2.au3"    
Window handle OK
UI Automation object OK
Automation element from window OK
Property condition OK
Find button ERR
>Exit code: 0    Time: 1.280
Link to comment
Share on other sites

I think I have found the error. When I use the Spy tool on the "Save Data..." button, I get this output:

 

Mouse position is retrieved 1200-706
At least we have an element [Save Data...][WindowsForms10.BUTTON.app.0.141b42a_r12_ad1]
Having the following values for all properties: 
Title is: <Save Data...>    Class   := <WindowsForms10.BUTTON.app.0.141b42a_r12_ad1>    controltype:= <UIA_ButtonControlTypeId> ,<50000>    , (00010502)    
*** Parent Information ***
Title is: <Data Search> Class   := <WindowsForms10.Window.8.app.0.141b42a_r12_ad1>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (000104E4)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=dataSearchB
UIA_BoundingRectanglePropertyId :=1151;691;102;31
UIA_ClassNamePropertyId :=WindowsForms10.BUTTON.app.0.141b42a_r12_ad1
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50000
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=WinForm
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=False
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Save Data...
UIA_LegacyIAccessibleRolePropertyId :=43
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=00100001
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=button
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Save Data...
UIA_NativeWindowHandlePropertyId :=00010502
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=4584
UIA_ProviderDescriptionPropertyId :=[pid:2960,hwnd:0x10502 Main:Nested [pid:4584,hwnd:0x10502 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=42;66818
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

The button is correctly reported with a type id of UIA_ButtonControlTypeId.

In your text box in post #176 the button is reported with a type id of UIA_PaneControlTypeId. This is wrong. The proper type is UIA_ButtonControlTypeId. Why is your button reported with a wrong type id?

Is the UI Automation API on your PC uptodate? Version 3.0. Are you using the newest versions of the UDFs in first post?

Edited by LarsJ
Link to comment
Share on other sites

Good eye LarsJ,

I have no idea why is reported with a wrong type id. 

I downloaded all files from post#1 by Junkew. There are 2 files with the same name, but Junkew says the smallest one is the most updated one with your corrections to the code. That is the one I downloaded. Is there any other update in a different post? I went trhough all pages before and don't recall seeing one. 

How can I check the UI Automation API version?

The Script Editor SciTE-Lite version is: Version 3.3.6 Dec.30 2013

Thanks for all your help.

Link to comment
Share on other sites

If you are on Windows 7 or 8 the API is OK. If you are on XP or Vista there may be a risk that the API isn't uptodate. See http://support.microsoft.com/kb/971513.

It's very important that you use AutoIt 3.3.10.2. Version 3.3.8 is flawed e.g. according to parameter transfer for some parameter types for the interface methods.

If you are using Scite (I'm not), I would recommend the full version.

You can use Inspect.exe from Windows SDK to check the output of the Spy tool. Inspect.exe also reports the "Save Data..." button with a type id of UIA_ButtonControlTypeId.

It's very very important that you figure out, why you are getting wrong control type ids. This means, that nothing will work.

If you have any questions, you can use this forum.

Regards Lars.

Link to comment
Share on other sites

Thanks a lot for your help LarsJ

I am on Windows 7. I don't think we have a problem there.

I downloaded the full version of Scite after your recomendation. I've been using that because I thougth that was the best option to write the script. What do you use? Should I use a different editor to write the script?

I found the inspect.exe tool from Windows and compare the output of the spy tool. The inspect.exe gives me the control type as a UIA_ButtonControlTypeId. Just like you are saying. I have inserted a picture of the output. 

 

I don't understand how the spy tool is giving a different control type than the inspect tool.

inspect.jpg

Link to comment
Share on other sites

LarsJ

Thanks for all your help. I want to apologize for the time invested. I am new in this automation world and very new in Autoit. I believe is powerful and amazing. I don't know if it matters the editor, and I will test later, but I copied your code to notepad++, save it, then right clicked on the file and compile it. This created the exe file, which runs as administrator and clicks the save button from the HOBO utility when is enabeled. 

I will follow your example to create the whole automation process for the utility, which is selecting the device, clicking the Get/Refresh button, then the Generate Data Streams, and finally the Save button.

Thanks a lot.

Link to comment
Share on other sites

casma, You're welcome. I'm only answering if I have the time for it. Good to see that you are able to go on with your project.

This has absolutely nothing to do with the editor. I'm using Scite myself to run code with F5 to see ConsoleWrite output. But I'm not using Scite for editing. For editing I'm using Notepad++. (This means that often I have both editors open.)

The Spy tool is definitely reliable. This is simply a consequence of the implementation. The Spy tool is just doing some kind of reverse lookup of the control ids and pattern ids (to get the names of the equivalent variable names) returned directly by the interface methods in the UI Automation DLL files.

You should download the UDFs and the Spy tool in the first post once more. Something must have happened with these files.

Regards Lars.

Link to comment
Share on other sites

I'll do that LarsJ

I've been working on the project and everything is running much smoother. I've been following the same logic as you did in your code. Find the buttons and creating a condition property, then find the button and then invoke the instance of the button. At least that's how I've tried to think of this. This is working perfect for all buttons.

But I run into another problem when trying to automate other elements of the application just like the type:

UIA_ListItemControlTypeId to select the devices  detected by the utility.

And once I select the "Save Data..." button the utility displays a "Save As" window where I can select the location and name of the file which contains the data. I am going to be overwritting the same file everytime I export the data and the field to type in the name of the file is: UIA_EditControlTypeId.

Any ideas how to work on this or a good example to reference this to. 

Thanks a lot for all your help. 

Link to comment
Share on other sites

Check what you want to use

The UIAWrappers will make life a little easier by abstracting stuff in a generic way

for example see example 11 

Example 9 is more flexible (in between low level use of the interfaces or fully abstracted as in example 11)

Edited by junkew
Link to comment
Share on other sites

Thanks a lot for your reply Junkew.

I've been trying 2 different approachs with no luck yet. I was hoping you guys could help me determine which is the best for this scenario.

The first one I tried is based on examples 11 and 9 as suggested by Junkew. Although if I try this I have to comment out Opt("MustDeclareVars",1) otherwise I get errors of variables not being declared in the UIAWrappers file. I have downloaded this one several times from post #1. Other errors are like:

"C:\Users\richi\Desktop\UIAWrappers.au3" (657) : ==> Variable must be of type "Object".:
$obj.FindAll($treescope, $oCondition, $pElements)
$obj^ ERROR
;Opt( "MustDeclareVars", 1 )

....

 Local $oSensor1
  _UIA_setVar("HOBO.sensor1","name:=416 SN:10485416; controltype:=ListItem" )
  $oSensor1=_UIA_getFirstObjectOfElement($hWindow,_UIA_getVar("HOBO.sensor2"),$treescope_subtree)
  _UIA_action("HOBO.sensor1","leftclick")

The other code I have tried is this one:

;Condition to find "416 SN: 10485416" sensor
  Local $pConditionSensor1, $pCondition1Sensor1, $pCondition2Sensor1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition1Sensor1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "416 SN: 10485416; controltype:=ListItem", $pCondition2Sensor1 )
  $oUIAutomation.CreateAndCondition( $pCondition1Sensor1, $pCondition2Sensor1, $pConditionSensor1 )
  If Not $pConditionSensor1 Then Return ConsoleWrite( "Property condition ERR Sensor1" & @CRLF )
  ConsoleWrite( "Property condition OK Sensor1" & @CRLF )


; Find "416 SN: 10485416" sensor
  Local $pListSensor1, $oListSensor1
  $oWindow.FindFirst( $TreeScope_Descendants, $pConditionSensor1, $pListSensor1 )
  $oListSensor1 = ObjCreateInterface( $pListSensor1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
 ; If Not IsObj( $oListSensor1 ) Then Return ConsoleWrite( "Find List ERR Sensor 1" & @CRLF )
 ; ConsoleWrite( "Find List OK Sensor 1" & @CRLF )


  ; Click (invoke) "416 SN: 10485416"
  Local $pInvokeSensor1, $oInvokeSensor1
  $oListSensor1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeSensor1 )
  $oInvokeSensor1 = ObjCreateInterface( $pInvokeSensor1, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokeSensor1) Then Return ConsoleWrite( "Invoke pattern ERR Sensor 1" & @CRLF )
  ConsoleWrite( "Invoke pattern OK Sensor 1" & @CRLF )
  $oInvokeSensor1.Invoke()

And the console returns this:

Window handle OK
UI Automation object OK
Automation element from window OK
Property condition OK Save
Property condition OK Time
Property condition OK Generate
Property condition OK Sensor1
"C:\Users\richi\Desktop\test_notepad.au3" (103) : ==> Variable must be of type "Object".:
$oListSensor1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeSensor1 )
$oListSensor1^ ERROR

or if I don't comment out condition for sensor:

Property condition OK Time
Property condition OK Generate
Property condition OK Sensor1
Find List ERR Sensor 1

Thanks a lot for taking a look at this. 

Link to comment
Share on other sites

Try this to select an item:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Get window handle
  Local $hWindow = WinGetHandle( "HOBOnode Viewer Utility" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; --- Find list control ---

  ; There are two list controls, we need the proper one

  ; Condition to find list controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find list controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find list controls ERR" & @CRLF )
  ConsoleWrite( "Find list controls OK" & @CRLF )

  ; Find the proper list control
  Local $pList, $oList, $sName
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pList )
    $oList = ObjCreateInterface( $pList, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oList.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sName )
    ConsoleWrite( "List control class name: " & $sName & @CRLF )
    If StringInStr( $sName, "LISTBOX" ) Then
      ConsoleWrite( "This is the proper list" & @CRLF )
      ExitLoop
    EndIf
  Next

  ; --- Select list item ---

  ; Condition to find list item
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "416 SN: 10485416", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find list item
  Local $pItem, $oItem
  $oList.FindFirst( $TreeScope_Descendants, $pCondition, $pItem )
  $oItem = ObjCreateInterface( $pItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oItem ) Then Return ConsoleWrite( "Find item ERR" & @CRLF )
  ConsoleWrite( "Find item OK" & @CRLF )

  ; Add item to selection
  Local $pSelection, $oSelection
  $oItem.GetCurrentPattern( $UIA_SelectionItemPatternId, $pSelection )
  $oSelection = ObjCreateInterface( $pSelection, $sIID_IUIAutomationSelectionItemPattern, $dtagIUIAutomationSelectionItemPattern )
  $oSelection.AddToSelection()

EndFunc
Link to comment
Share on other sites

  • 2 weeks later...

Hi.

This looks amazing.  I'm trying to get the state of a radio button in a dialog and nothing in default AutoIt is working.  Here's the Spy output of the control:

Mouse position is retrieved 386-359
At least we have an element [Items][WindowsForms10.BUTTON.app.0.165f26b_r59_ad1]
Having the following values for all properties: 
Title is: <Items>   Class   := <WindowsForms10.BUTTON.app.0.165f26b_r59_ad1>    controltype:= <UIA_RadioButtonControlTypeId>    ,<50013>    , (0000C35D)    
*** Parent Information ***
Title is: <>    Class   := <WindowsForms10.Window.8.app.0.165f26b_r59_ad1>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=radioButton_Output_Items
UIA_BoundingRectanglePropertyId :=365;356;50;18
UIA_ClassNamePropertyId :=WindowsForms10.BUTTON.app.0.165f26b_r59_ad1
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50013
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=WinForm
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=True
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=True
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=Check
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Items
UIA_LegacyIAccessibleRolePropertyId :=45
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=1048596
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=radio button
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Items
UIA_NativeWindowHandlePropertyId :=66296
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=1568
UIA_ProviderDescriptionPropertyId :=[pid:5360,hwnd:0x102F8 Main:Nested [pid:1568,hwnd:0x102F8 Main(parent link):Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.DLL)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=42;66296
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=True
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Can someone show me simple code to check whether this radio button is checked or not?

Many thanks.

Link to comment
Share on other sites

Try this:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  Local $hWindow = WinGetHandle(  ) ; <<<< Fill in window info <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find radio button
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_RadioButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Items", $pCondition2 ) ; <<<< Name <<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find radio button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "Find button ERR" & @CRLF )
  ConsoleWrite( "Find button OK" & @CRLF )

  ; Get selected state
  Local $fSelected
  $oButton.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $fSelected )
  ConsoleWrite( "Selected: " & $fSelected & @CRLF )

EndFunc
Link to comment
Share on other sites

I'm trying to open Action Center so I can set the state of the checkboxes inside.

It seems I can get the control ID with the below but the "_UIA_action($control, "Left", 0 , 0, 0) ; leftclick" always results in an error.

I've not modified the includes, and the error I see is about an array which is malformed. MustDeclareVars has to be set to 0 for the below to work.

If someone could point out an example that shows how to get a handle on the control and click it I might be able to figure it out from there.

#include "CUIAutomation2.au3"
#include <uiawrappers.au3>

Opt( "MustDeclareVars", 0 )

Global $oUIAutomation
Run("rundll32.exe shell32.dll,Control_RunDLL wscui.cpl", "", @SW_HIDE)

WinWait("Action Center")
WinActivate("Action Center")
WinWaitActive("Action Center")
MainFunc()


Func MainFunc()

;~   Local $hWindow = WinGetHandle( "[CLASS:Element]" )              ;
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )              ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )             ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )       ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[TITLE:Test; CLASS:AutoIt v3 GUI]" ) ; AutoIt GUI window
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                   ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "Calculator" )                        ; Calculator
  If Not $hWindow Then
    MsgBox(4096,"","")
    Return
  EndIf

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  ;$oUIAutomation.GetRootElement( $pWindow )             ; Desktop
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow ) ; Window
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ;ListDescendants( $oWindow, 0, 1 ) ; Desktop
  ListDescendants( $oWindow, 0, 0 )  ; Window

EndFunc


Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )

  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  Local $pRawWalker, $oRawWalker
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )

  Local $pUIElement, $oUIElement
  $oRawWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
;~     ConsoleWrite( $sIndent & "Title     = " & _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
;~                   $sIndent & "Class     = " & _UIA_getPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
;~                   $sIndent & "Ctrl type = " & _UIA_getPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
;~                   $sIndent & "Ctrl name = " & _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
;~                   $sIndent & "Selected  = " & _UIA_getPropertyValue( $oUIElement, $UIA_SelectionItemIsSelectedPropertyId ) & @CRLF & _
;~                   $sIndent & "Handle    = " & Hex( _UIA_getPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
;~ ControlClick(_UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ),"", _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ))
;~ ControlClick ( "Address: Control Panel\All Control Panel Items\Action Center", "Action Center", "00000000")


Local $title = _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId )
If $title = "Action Center" Then
;~ MsgBox(4096,"",$title)
Local $control = hex(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId))
;~ MsgBox(4096,"",$control, 1)
_UIA_action($control, "Left", 0 , 0, 0) ; leftclick
;~ ControlClick("Action Center","",$control)
EndIf

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd

EndFunc

Always carry a towel.

Link to comment
Share on other sites

Try this:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; --- Change Action Center settings ---

  ; Get window handle
  Local $hWindow = WinGetHandle( "[TITLE:Action Center; CLASS:CabinetWClass]", "")
  If Not $hWindow Then      ; With this If-statement, you can run the script several
    ; Open "Action Center"  ; times without the need to close "Action Center".
    Run( "rundll32.exe shell32.dll,Control_RunDLL wscui.cpl" )
    WinWait( "Action Center" )
    WinActivate( "Action Center" )
    WinWaitActive( "Action Center" )
    $hWindow = WinGetHandle( "[TITLE:Action Center; CLASS:CabinetWClass]", "")
  EndIf
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find link
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Change Action Center settings", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find link
  Local $pLink, $oLink
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
  $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oLink ) Then Return ConsoleWrite( "Find Link ERR" & @CRLF )
  ConsoleWrite( "Find Link OK" & @CRLF )

  ; Click (invoke) link
  Local $pInvoke, $oInvoke
  $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "Invoke pattern ERR" & @CRLF )
  ConsoleWrite( "Invoke pattern OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )

  ; --- Click "Windows Update" checkbox ---

  ; Get window handle
  Local $hWindow = WinGetHandle( "[TITLE:Change Action Center settings; CLASS:CabinetWClass]", "")
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find checkbox
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CheckBoxControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Windows Update", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find checkbox
  Local $pCheckbox, $oCheckbox
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pCheckbox )
  $oCheckbox = ObjCreateInterface( $pCheckbox, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oCheckbox ) Then Return ConsoleWrite( "Find Checkbox ERR" & @CRLF )
  ConsoleWrite( "Find Checkbox OK" & @CRLF )

  ; Toggle checkbox
  Local $pToggle, $oToggle
  $oCheckbox.GetCurrentPattern( $UIA_TogglePatternId, $pToggle )
  $oToggle = ObjCreateInterface( $pToggle, $sIID_IUIAutomationTogglePattern, $dtagIUIAutomationTogglePattern )
  If Not IsObj( $oToggle ) Then Return ConsoleWrite( "Toggle pattern ERR" & @CRLF )
  ConsoleWrite( "Toggle pattern OK" & @CRLF )
  $oToggle.Toggle()

  ; Get toggle state
  Local $fToggle
  $oCheckbox.GetCurrentPropertyValue( $UIA_ToggleToggleStatePropertyId, $fToggle )
  ConsoleWrite( "Toggle: " & $fToggle & " <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<" & @CRLF )

  ; --- Click OK button ---

  ; Condition to find OK button
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "OK", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find OK button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "Find button ERR" & @CRLF )
  ConsoleWrite( "Find button OK" & @CRLF )

  ; Click (invoke) OK button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "Invoke pattern ERR" & @CRLF )
  ConsoleWrite( "Invoke pattern OK" & @CRLF )
  $oInvoke.Invoke()

EndFunc
Edited by LarsJ
Link to comment
Share on other sites

Try this:

 

Wow, that worked perfectly, and is more than I could have figured out from a hint. I will study this to see if I can use it on other windows. Thank you!

I made a hybrid from it with a fixed function to check the box instead of toggle, it's less code for all the items and they are accessible to ControlFocus/ControlCommand.

#include "CUIAutomation2.au3"

Opt("MustDeclareVars", 1)

MainFunc()

Func MainFunc()

    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("UI Automation object ERR" & @CRLF)
    ConsoleWrite("UI Automation object OK" & @CRLF)

    ; --- Change Action Center settings ---

    ; Get window handle
    Local $hWindow = WinGetHandle("[TITLE:Action Center; CLASS:CabinetWClass]", "")
    If Not $hWindow Then ; With this If-statement, you can run the script several
        ; Open "Action Center"  ; times without the need to close "Action Center".
        Run("rundll32.exe shell32.dll,Control_RunDLL wscui.cpl")
        WinWait("Action Center")
        WinActivate("Action Center")
        WinWaitActive("Action Center")
        $hWindow = WinGetHandle("[TITLE:Action Center; CLASS:CabinetWClass]", "")
    EndIf
    If Not $hWindow Then Return ConsoleWrite("Window handle ERR" & @CRLF)
    ConsoleWrite("Window handle OK" & @CRLF)

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow)
    $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return ConsoleWrite("Automation element from window ERR" & @CRLF)
    ConsoleWrite("Automation element from window OK" & @CRLF)

    ; Condition to find link
    Local $pCondition, $pCondition1, $pCondition2
    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition1)
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Change Action Center settings", $pCondition2)
    $oUIAutomation.CreateAndCondition($pCondition1, $pCondition2, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("Property condition ERR" & @CRLF)
    ConsoleWrite("Property condition OK" & @CRLF)

    ; Find link
    Local $pLink, $oLink
    $oWindow.FindFirst($TreeScope_Descendants, $pCondition, $pLink)
    $oLink = ObjCreateInterface($pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oLink) Then Return ConsoleWrite("Find Link ERR" & @CRLF)
    ConsoleWrite("Find Link OK" & @CRLF)

    ; Click (invoke) link
    Local $pInvoke, $oInvoke
    $oLink.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
    ConsoleWrite("Invoke pattern OK" & @CRLF)
    $oInvoke.Invoke()
    Sleep(1000)

    ControlFocus("Change Action Center settings", "", "Button1")
    ControlCommand("Change Action Center settings", "", "Button1", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button2")
    ControlCommand("Change Action Center settings", "", "Button2", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button3")
    ControlCommand("Change Action Center settings", "", "Button3", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button4")
    ControlCommand("Change Action Center settings", "", "Button4", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button5")
    ControlCommand("Change Action Center settings", "", "Button5", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button6")
    ControlCommand("Change Action Center settings", "", "Button6", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button7")
    ControlCommand("Change Action Center settings", "", "Button7", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button8")
    ControlCommand("Change Action Center settings", "", "Button8", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button9")
    ControlCommand("Change Action Center settings", "", "Button9", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button10")
    ControlClick("Change Action Center settings", "", "Button10")

    Sleep(100)
    WinClose($hWindow, "")

;~  CabinetWClass
EndFunc   ;==>MainFunc

Always carry a towel.

Link to comment
Share on other sites

Updated first post with references to all (working) examples in thread

Added 2 zip files to first post. 

UIA_V0_4.ZIP with all au3 modules to handle IUIAutomation, IAccessible, IAccessible2 and ISimpleDom

EXAMPLES.ZIP containing about 30 examples as are discussed in this whole thread 

Enhanced the UIAWrappers.AU3 with some logic (for easier testautomation and some abstraction of interfaces)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...