noviTester Posted May 19, 2015 Share Posted May 19, 2015 (edited) Hello,I apologize if the subject has already opened, haven't find it anywhere.I'm trying to automate an application which has ribbon menu.The Finder tool from AutoIt Window Info sees only entire ribbon menu. Only way to click on the different ribbon menu item is using position (coordinates), which is not acceptable. Is there any way to get IDs of ribbons?Thanks in advance! Edited May 26, 2015 by noviTester Link to comment Share on other sites More sharing options...
Sandy89 Posted October 27, 2016 Share Posted October 27, 2016 Hello, I have the same question. is it still unanswered? Link to comment Share on other sites More sharing options...
beeria Posted October 27, 2016 Share Posted October 27, 2016 you can try keyboard shortcut if there is any Link to comment Share on other sites More sharing options...
Sandy89 Posted October 28, 2016 Share Posted October 28, 2016 there is no shortcut keys in this tool which I am trying to automate. Link to comment Share on other sites More sharing options...
LarsJ Posted October 28, 2016 Share Posted October 28, 2016 A ribbon can be automated with the UI Automation framework. This is an example with WordPad. Start to open WordPad. No need to open a file. Get an overview of the controls in the ribbon. Here the entire window: expandcollapse popup#include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Global $oUIAutomation Example() ; List all automation elements of a window ; in a hierarchical structure like a treeview. Func Example() ; Get window handle ;Local $hWindow = 0x0006045C ; Infragistics ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Infragistics ;Local $hWindow = WinGetHandle( "[REGEXPCLASS:WindowsForms10.Window.8.app.*]" ) ; Windows Updates Downloader ;Local $hWindow = WinGetHandle( "BCGPVisualStudioGUIDemo - Start Page" ) ; BCGSoft BCGPVisualStudioGUIDemo ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" ) ; Windows Explorer, Windows 7 ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" ) ; Windows Explorer, Windows XP ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" ) ; Windows Explorer right pane ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" ) ; AutoIt GUI window ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" ) ; AutoIt script ;Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" ) ; Chrome ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" ) ; Internet Explorer Local $hWindow = WinGetHandle( "[CLASS:WordPadClass]" ) ; WordPad ;Local $hWindow = WinGetHandle( "Calculator" ) ; Calculator If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF ) ConsoleWrite( "$hWindow OK" & @CRLF ) ; Create UI Automation object $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get UI Automation element from window handle Local $pWindow, $oWindow $oUIAutomation.ElementFromHandle( $hWindow, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF & @CRLF ) ; List all elements of window ListDescendants( $oWindow, 0 ) EndFunc ; List all child elements of parent Func ListDescendants( $oParent, $iLevel, $iLevels = 0, $sIndent = "" ) If Not IsObj( $oParent ) Then Return If $iLevels And $iLevel = $iLevels Then Return ; Create RawViewWalker object Local $pRawViewWalker, $oRawViewWalker $oUIAutomation.RawViewWalker( $pRawViewWalker ) $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker ) If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF ) ; Get first child element Local $pUIElement, $oUIElement $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement ) $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) For $i = 0 To $iLevel - 1 $sIndent &= " " Next While IsObj( $oUIElement ) ConsoleWrite( $sIndent & "Title = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _ $sIndent & "Class = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _ $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _ $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _ $sIndent & "Value = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _ $sIndent & "Handle = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF ) ListDescendants( $oUIElement, $iLevel + 1, $iLevels ) $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement ) $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) WEnd EndFunc Func GetCurrentPropertyValue( $oObject, $iPropertyId ) Local $vValue, $sString $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue ) If Not IsArray( $vValue ) Then Return $vValue $sString = $vValue[0] For $i = 1 To UBound( $vValue ) - 1 $sString &= "; " & $vValue[$i] Next Return $sString EndFunc The output in SciTE console should look like this: expandcollapse popup$hWindow OK $oUIAutomation OK $oWindow OK Title = UIRibbonDockTop Class = UIRibbonCommandBarDock Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103D6 Title = Ribbon Class = UIRibbonCommandBar Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103DC Title = Ribbon Class = UIRibbonWorkPane Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103E4 Title = Class = NUIPane Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103CE Title = Ribbon Class = NetUIHWND Ctrl type = 50033 Ctrl name = pane Value = Ribbon Handle = 000103D0 Title = Class = NetUICtrlNotifySink Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103DE Title = Font family Class = RICHEDIT50W Ctrl type = 50004 Ctrl name = edit Value = Calibri Handle = 00010408 Title = Class = NetUICtrlNotifySink Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103E0 Title = Font size Class = RICHEDIT50W Ctrl type = 50004 Ctrl name = edit Value = 11 Handle = 0001040A Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Quick Access Toolbar Class = Ctrl type = 50021 Ctrl name = tool bar Value = Quick Access Toolbar Handle = 00000000 Title = Save Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Undo Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Redo Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Customize Quick Access Toolbar Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Application menu Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Ribbon tabs Class = Ctrl type = 50018 Ctrl name = tab Value = Ribbon tabs list Handle = 00000000 Title = Class = Ctrl type = 50025 Ctrl name = Value = Handle = 00000000 Title = Home Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = View Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Help Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Lower Ribbon Class = Ctrl type = 50033 Ctrl name = pane Value = Lower Ribbon Handle = 00000000 Title = Class = Ctrl type = 50025 Ctrl name = Value = Handle = 00000000 Title = Home Class = Ctrl type = 50025 Ctrl name = Value = Ribbon tab Handle = 00000000 Title = Clipboard Class = Ctrl type = 50021 Ctrl name = tool bar Value = Group Handle = 00000000 Title = Paste Class = Ctrl type = 50025 Ctrl name = Value = Handle = 00000000 Title = Paste Class = Ctrl type = 50031 Ctrl name = split button Value = Paste (Ctrl+V) Handle = 00000000 Title = Paste Class = Ctrl type = 50031 Ctrl name = split button Value = Handle = 00000000 Title = Cut Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Copy Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Font Class = Ctrl type = 50021 Ctrl name = tool bar Value = Group Handle = 00000000 Title = Font family Class = Ctrl type = 50003 Ctrl name = combo box Value = Calibri Handle = 00000000 Title = Open Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Font size Class = Ctrl type = 50003 Ctrl name = combo box Value = 11 Handle = 00000000 Title = Open Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Grow font Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Shrink font Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Bold Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Italic Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Underline Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Strikethrough Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Subscript Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Superscript Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Text color Class = Ctrl type = 50025 Ctrl name = Value = Automatic Handle = 00000000 Title = Text color Class = Ctrl type = 50031 Ctrl name = split button Value = Automatic Handle = 00000000 Title = Text color Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Text highlight color Class = Ctrl type = 50025 Ctrl name = Value = No color Handle = 00000000 Title = Text highlight color Class = Ctrl type = 50031 Ctrl name = split button Value = No color Handle = 00000000 Title = Text highlight color Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Paragraph Class = Ctrl type = 50021 Ctrl name = tool bar Value = Group Handle = 00000000 Title = Decrease indent Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Increase indent Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Lists Class = Ctrl type = 50025 Ctrl name = Value = Handle = 00000000 Title = Lists Class = Ctrl type = 50031 Ctrl name = split button Value = Handle = 00000000 Title = Lists Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Change line spacing Class = Ctrl type = 50031 Ctrl name = split button Value = Handle = 00000000 Title = Align text left Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Align text center Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Align text right Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Justify text Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Paragraph Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Insert Class = Ctrl type = 50021 Ctrl name = tool bar Value = Group Handle = 00000000 Title = Picture Class = Ctrl type = 50025 Ctrl name = Value = Handle = 00000000 Title = Picture Class = Ctrl type = 50031 Ctrl name = split button Value = Insert picture Handle = 00000000 Title = Picture Class = Ctrl type = 50031 Ctrl name = split button Value = Handle = 00000000 Title = Paint drawing Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Date and time Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Insert object Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Editing Class = Ctrl type = 50021 Ctrl name = tool bar Value = Group Handle = 00000000 Title = Find Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Replace Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Select all Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Class = Afx:00000000FF0E0000:0:0000000000010003:0000000000000010:0000000000000000 Ctrl type = 50033 Ctrl name = pane Value = Handle = 000303AE Title = Class = msctls_statusbar32 Ctrl type = 50017 Ctrl name = status bar Value = Handle = 000203B2 Title = Class = Afx:00000000FF0E0000:0 Ctrl type = 50033 Ctrl name = pane Value = Handle = 000203B0 Title = 100% Class = Static Ctrl type = 50020 Ctrl name = text Value = Handle = 000103B8 Title = - Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000103BA Title = Zoom Slider Class = msctls_trackbar32 Ctrl type = 50015 Ctrl name = slider Value = 50 Handle = 000103BC Title = Page left Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Position Class = Ctrl type = 50027 Ctrl name = thumb Value = Handle = 00000000 Title = Page right Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = + Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000103BE Title = Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = Class = Afx:00000000FF0E0000:0:0000000000010003:0000000000000010:0000000000000000 Ctrl type = 50033 Ctrl name = pane Value = Handle = 000103C8 Title = Rich Text Window Class = RICHEDIT50W Ctrl type = 50004 Ctrl name = edit Value = Handle = 000303A4 Title = Class = Ctrl type = 50037 Ctrl name = title bar Value = Document - WordPad Handle = 00000000 Title = System Class = Ctrl type = 50010 Ctrl name = menu bar Value = Handle = 00000000 Title = System Class = Ctrl type = 50011 Ctrl name = menu item Value = Handle = 00000000 Title = Minimize Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Maximize Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Close Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Click the split button to create a list: expandcollapse popup#include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Global $oUIAutomation Example() Func Example() Local $hWindow = WinGetHandle( "[CLASS:WordPadClass]" ) If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF ) ConsoleWrite( "$hWindow OK" & @CRLF ) ; Create UI Automation object $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get UI Automation element from window handle Local $pWindow, $oWindow $oUIAutomation.ElementFromHandle( $hWindow, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF ) ; Condition to find split button Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_SplitButtonControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Condition to find "Lists" ; "Lists" is a localized word ; Use the word in your own language Local $pCondition2 $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Lists", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition ; Both conditions Local $pCondition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) ; Find "Lists" split button Local $pButton, $oButton $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton ) $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF ) ConsoleWrite( "$oButton OK" & @CRLF ) ; Click (Invoke) button Local $pInvoke, $oInvoke $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndFunc mLipok and LudwigIt 2 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted October 28, 2016 Share Posted October 28, 2016 Sorry, I am new to AutoIT. the tool which I wanna automate is not from Microsoft. So will this way would be effective? Also which version is needed if we need to use the UDF CUIAutomation2.au3? Link to comment Share on other sites More sharing options...
LarsJ Posted October 28, 2016 Share Posted October 28, 2016 Effective: Yes. AutoIt version: 3.3.10+. CUIAutomation2.au3 version: Newest. Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted October 28, 2016 Share Posted October 28, 2016 I installed newest version (3.3.14.2), still I couldn't find CUIAutomation2.au3 in include folder. Link to comment Share on other sites More sharing options...
LarsJ Posted October 28, 2016 Share Posted October 28, 2016 (edited) You have to download CUIAutomation2.au3 from the UI Automation framework. Zip in bottom of first post. The include folder only contains the official AutoIt UDFs. Edited October 28, 2016 by LarsJ Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted November 1, 2016 Share Posted November 1, 2016 (edited) it worked for me thanks a lot for your valuable time. but my bad luck, the control ID and handle of the item I want to click has same values with other items in the same ribbon. Title = 10SAE Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = File System Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = PAR Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 I want to click the item "PAR". how can it be done? Edited November 1, 2016 by Sandy89 Link to comment Share on other sites More sharing options...
LarsJ Posted November 1, 2016 Share Posted November 1, 2016 What are these controls? Are they items in a menu? Is it possible for you to show the structure of the entire ribbon as I have done in post 5? (Click the "<>" button when you are editing the post and select "plain" to insert plain text.) Handle = 0 means that the controls have no handle. If you search for 50020 in CUIAutomation2.au3 you'll find Global Const $UIA_TextControlTypeId=50020 The controls are text controls. You can use code like this to identify the text control ; Condition to find text control Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Condition to find "PAR" Local $pCondition2 $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "PAR", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition ; Both conditions Local $pCondition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) If the controls are menu items you would not click the text control, but the parent menu item. Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted November 1, 2016 Share Posted November 1, 2016 PAR will appear as text in this window, and if we click on that there will be drop down. I need to work on menu item as well in this ribbon ("Assign sources to datasets"). window containing PAR is under "Find sources" list item this is the structure of the entire ribbon expandcollapse popup$hWindow OK $oUIAutomation OK $oWindow OK Title = OK Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000C100C Title = Cancel Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000A1008 Title = Class = Static Ctrl type = 50006 Ctrl name = image Value = Handle = 00140FF6 Title = Class = SysListView32 Ctrl type = 50008 Ctrl name = list Value = Handle = 002806BA Title = Find Sources Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = Assign sources to datasets Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = Source Activation Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = Data Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = Deliver Options Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = Help Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 00270AE6 Title = Class = RICHEDIT Ctrl type = 50004 Ctrl name = edit Value = Find Sources Handle = 0038108E Title = Class = #32770 Ctrl type = 50033 Ctrl name = dialog Value = Handle = 0026084C Title = Class = SysListView32 Ctrl type = 50008 Ctrl name = list Value = Handle = 001D07CE Title = Header Control Class = SysHeader32 Ctrl type = 50034 Ctrl name = header Value = Handle = 00070FE8 Title = Name Class = Ctrl type = 50035 Ctrl name = header item Value = Handle = 00000000 Title = Origin Class = Ctrl type = 50035 Ctrl name = header item Value = Handle = 00000000 Title = Class Class = Ctrl type = 50035 Ctrl name = header item Value = Handle = 00000000 Title = Class = Ctrl type = 50007 Ctrl name = list item Value = Handle = 00000000 Title = 10SAE Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = File System Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = PAR Class = Ctrl type = 50020 Ctrl name = text Value = Handle = 00000000 Title = Browse File... Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 003206D2 Title = Pre-Merge... Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000B0FEE Title = Remove Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000F07A0 Title = Selected sources: Class = Static Ctrl type = 50020 Ctrl name = text Value = Handle = 003907F6 Title = Tab1 Class = SysTabControl32 Ctrl type = 50018 Ctrl name = tab Value = Handle = 001711E0 Title = Class = #32770 Ctrl type = 50033 Ctrl name = dialog Value = Handle = 000D1148 Title = GridControl Class = BCGPGridCtrl:c170000:8:10003:10 Ctrl type = 50036 Ctrl name = table Value = Handle = 00091016 Title = General Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Functions Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Parameters Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Object Names Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Attachments Class = Ctrl type = 50019 Ctrl name = tab item Value = Handle = 00000000 Title = Selected sources: Class = Static Ctrl type = 50020 Ctrl name = text Value = Handle = 0027102E Title = Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000D08BE Title = Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 003C0C18 Title = Open Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 00080FFA Title = Excel Import Class = Button Ctrl type = 50000 Ctrl name = button Value = Handle = 000D1138 Title = Class = Ctrl type = 50037 Ctrl name = title bar Value = Data Handle = 00000000 Title = System Class = Ctrl type = 50010 Ctrl name = menu bar Value = Handle = 00000000 Title = System Class = Ctrl type = 50011 Ctrl name = menu item Value = Handle = 00000000 Title = Minimize Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Restore Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Title = Close Class = Ctrl type = 50000 Ctrl name = button Value = Handle = 00000000 Link to comment Share on other sites More sharing options...
LarsJ Posted November 1, 2016 Share Posted November 1, 2016 Try this to click PAR: expandcollapse popup#include "CUIAutomation2.au3" Opt( "MustDeclareVars", 1 ) Global $oUIAutomation Example() Func Example() Local $hWindow = WinGetHandle( "[CLASS:WordPadClass]" ) ; <<<<<<<<<<<<<<<< Edit <<<<<<<<<<<<<<<< If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF ) ConsoleWrite( "$hWindow OK" & @CRLF ) ; Create UI Automation object $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation ) If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF ) ConsoleWrite( "$oUIAutomation OK" & @CRLF ) ; Get UI Automation element from window handle Local $pWindow, $oWindow $oUIAutomation.ElementFromHandle( $hWindow, $pWindow ) $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF ) ConsoleWrite( "$oWindow OK" & @CRLF ) ; Condition to find text control Local $pCondition1 $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition1 ) If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF ) ConsoleWrite( "$pCondition1 OK" & @CRLF ) ; Condition to find "PAR" Local $pCondition2 $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "PAR", $pCondition2 ) If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF ) ConsoleWrite( "$pCondition2 OK" & @CRLF ) ; And condition ; Both conditions Local $pCondition $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition ) If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF ) ConsoleWrite( "$pCondition OK" & @CRLF ) ; Find "PAR" control Local $pPAR, $oPAR $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pPAR ) $oPAR = ObjCreateInterface( $pPAR, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) If Not IsObj( $oPAR ) Then Return ConsoleWrite( "$oPAR ERR" & @CRLF ) ConsoleWrite( "$oPAR OK" & @CRLF ) ; Click (Invoke) PAR Local $pInvoke, $oInvoke $oPAR.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke ) $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern ) If Not IsObj( $oInvoke ) Then ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "Perform mouse click" & @CRLF ) Local $aBoundingRectangle $oPAR.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aBoundingRectangle ) Local $x = $aBoundingRectangle[0] + ( $aBoundingRectangle[2] - $aBoundingRectangle[0] ) / 2 Local $y = $aBoundingRectangle[1] + ( $aBoundingRectangle[3] - $aBoundingRectangle[1] ) / 2 WinActivate( $hWindow ) MouseClick( "primary", $x, $y ) Else ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndIf EndFunc Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted November 1, 2016 Share Posted November 1, 2016 it didn't work I got the output as below. $hWindow OK $oUIAutomation OK $oWindow OK $pCondition1 OK $pCondition2 OK $pCondition OK $oPAR OK $oInvoke ERR Perform mouse click is the searching happening with parent menu ("Find Sources") as base? we have list item and richedit with the name "Find Sources". Link to comment Share on other sites More sharing options...
LarsJ Posted November 1, 2016 Share Posted November 1, 2016 Is there more than one text control with the text "PAR" in the window? Can you show a picture of the ribbon? (Insert other media.) Will you add these commands before "WinActivate...": ConsoleWrite( "$x = " & $x & @CRLF ) ConsoleWrite( "$y = " & $y & @CRLF ) Does $x, $y match "PAR"? Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Sandy89 Posted November 2, 2016 Share Posted November 2, 2016 (edited) only one text control is present with "PAR". the coordinates does not match. this is the screenshot of the ribbon with text control "PAR". Edited November 2, 2016 by Sandy89 Link to comment Share on other sites More sharing options...
Sandy89 Posted November 2, 2016 Share Posted November 2, 2016 only one text control is present with "PAR". the coordinates does not match. this is the screenshot of the ribbon with text control "PAR". Link to comment Share on other sites More sharing options...
Sandy89 Posted November 2, 2016 Share Posted November 2, 2016 is it possible to capture this image and click on the image each time to trigger this text control? Link to comment Share on other sites More sharing options...
Sandy89 Posted November 2, 2016 Share Posted November 2, 2016 when I edited like below, it is choosing the correct item. If Not IsObj( $oInvoke ) Then ConsoleWrite( "$oInvoke ERR" & @CRLF ) ConsoleWrite( "Perform mouse click" & @CRLF ) Local $aBoundingRectangle $oPAR.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aBoundingRectangle ) Local $x = $aBoundingRectangle[0] + 10 Local $y = $aBoundingRectangle[1] + 10 ConsoleWrite( "$x = " & $x & @CRLF ) ConsoleWrite( "$y = " & $y & @CRLF ) WinActivate( "[CLASS:#32770]" ) MouseClick( "primary", $x, $y ) Else ConsoleWrite( "$oInvoke OK" & @CRLF ) $oInvoke.Invoke() EndIf but why invoke function is not getting triggered? is it possible to capture this image and click on the image each time to trigger this text control? thanks a lot LarsJ!! Link to comment Share on other sites More sharing options...
LarsJ Posted November 2, 2016 Share Posted November 2, 2016 I was going to add this answer to posts 16 - 18: The "PAR" text control seems to be a cell in a listview. And it seems as if the listview only contains one row. Normally a row in a listview is about 20 pixels high, but this row looks like it's several hundred pixels high. "$oPAR OK" tells that the "PAR" text control is properly identified. We just have to figure out how to click it. Will you add these lines and check if the bounding rectangle of the "PAR" control fits: ConsoleWrite( "Left = " & aBoundingRectangle[0] & @CRLF ) ConsoleWrite( "Top = " & aBoundingRectangle[1] & @CRLF ) ConsoleWrite( "Right = " & aBoundingRectangle[2] & @CRLF ) ConsoleWrite( "Bottom = " & aBoundingRectangle[3] & @CRLF ) Will you copy/paste the values so I can see them. Where do you click the PAR control when you click it? Do you click exactly on the "PAR" text or do you just click somewhere in this large cell. The code in post 13 clicks in the middle of the cell. But this seems to be wrong. Now I can see you have already figured it out. The Invoke method only works for a control if code is added to the control to support it. This code is not added to your listview cell. The Invoke method is normally used for buttons and similar controls. You can make a screen dump of the window, identify the "PAR"-text with OCR-software, calculate x,y-coordinates and click it. But UI Automation code that directly identifies the control and generates the bounding rectangle is much better. And you can use the same method for similar ribbon controls. Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now