IgorCabrera Posted October 1, 2019 Posted October 1, 2019 Greetings Autoit Experts, I’m a newbie in automation and I have to automate OBS Studio (The main goal of my project is to integrate it to MS Powerpoint using VBA macros to change OBS scenes every time next slide is presented). For that I need to iterate with List Boxes, Buttons and ComboBoxes of the UI The case is that OBS 23 and OBS 24 (Last 2 versions) use QT controls that have elements in them (I read that it is possible to automate them but I cannot find how to). Every control in the form is an instance of Qt5QWindowIcon class with many elements. And The instance of the controls are not always the same. 😐 As an example, I need to click Sources setup button, but it is an element of a Qt5QWindowIcon control, in Autoit v3 Window Info you see this data: >>>> Control <<<< Class: Qt5QWindowIcon Instance: 4 ClassnameNN: Qt5QWindowIcon4 Name: Advanced (Class): [CLASS:Qt5QWindowIcon; INSTANCE:4] ID: Text: obs64 Position: 172, 274 Size: 281, 145 ControlClick Coords: 68, 37 Style: 0x56000000 ExStyle: 0x00000000 Handle: 0x000A0BC6 ******************* And in RanorexSpy : >>> Element <<< Role: PushButton (43) Name: Propiedades Class: Qt5QWindowIcon
IgorCabrera Posted October 1, 2019 Author Posted October 1, 2019 This is SimpleSpy message: Quote https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 At least we have an element title: [Propiedades] class: [] Having the following values for all properties: Title is: <Propiedades> Class := <> controltype:= <UIA_ButtonControlTypeId> ,<50000> , (0000C350) 633;421;26;25 *** Parent Information top down *** 4: Title is: <OBS 23.0.2 (64-bit, windows) - Perfíl: Sin Título - Escenas: Sin Título> Class := <Qt5QWindowIcon> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 395;2;892;483 "Title:=OBS 23.0.2 (64-bit, windows) - Perfíl: Sin Título - Escenas: Sin Título;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon"" 3: Title is: <Fuentes> Class := <Qt5QWindowIcon> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 575;307;281;145 "Title:=Fuentes;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon"" 2: Title is: <> Class := <> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 575;328;281;124 "Title:=;controltype:=UIA_CustomControlTypeId;class:="" 1: Title is: <> Class := <> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 579;332;273;116 "Title:=;controltype:=UIA_CustomControlTypeId;class:="" 0: Title is: <> Class := <> controltype:= <UIA_ToolBarControlTypeId> ,<50021> , (0000C365) 580;420;271;27 "Title:=;controltype:=UIA_ToolBarControlTypeId;class:="" ;~ *** Standard code maintainable *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) _UIA_setVar("oP1","Title:=OBS 23.0.2 (64-bit, windows) - Perfíl: Sin Título - Escenas: Sin Título;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon") ;OBS 23.0.2 (64-bit, windows) - Perfíl: Sin Título - Escenas: Sin Título _UIA_setVar("oP2","Title:=Fuentes;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon") ;Fuentes _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP4","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP5","Title:=;controltype:=UIA_ToolBarControlTypeId;class:=") ; ;~ $oUIElement=_UIA_getObjectByFindAll("Propiedades.mainwindow", "title:=Propiedades;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree) _UIA_setVar("oUIElement","Title:=Propiedades;controltype:=UIA_ButtonControlTypeId;class:=") ;ControlType:=UIA_ButtonControlTypeId;classname:=") ;~ Actions split away from logical/technical definition above can come from configfiles ;~_UIA_Action("oP1","highlight") _UIA_Action("oP1","setfocus") ;~_UIA_Action("oP2","highlight") _UIA_Action("oP2","setfocus") ;~_UIA_Action("oP3","highlight") _UIA_Action("oP3","setfocus") ;~_UIA_Action("oP4","highlight") _UIA_Action("oP4","setfocus") ;~_UIA_Action("oP5","highlight") _UIA_Action("oP5","setfocus") _UIA_action("oUIElement","highlight") ;~_UIA_action("oUIElement","click") ;~ *** Standard code Flexible*** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=OBS 23.0.2 (64-bit, windows) - Perfíl: Sin Título - Escenas: Sin Título;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=Fuentes;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon", $treescope_children) _UIA_Action($oP3,"setfocus") Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_ToolBarControlTypeId;class:=", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll("Propiedades.mainwindow", "title:=Propiedades;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Propiedades;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree) ;~_UIA_action($oUIElement,"highlight") _UIA_action($oUIElement,"click") *** Detailed properties of the highlighted element *** UIA_title:= <Propiedades> UIA_text:= <Propiedades> UIA_regexptitle:= <Propiedades> UIA_iaccessiblechildId:= <0> UIA_handle:= <0> UIA_BoundingRectangle:= <633;421;26;25> UIA_ProcessId:= <13052> UIA_ControlType:= <50000> UIA_LocalizedControlType:= <botón> UIA_Name:= <Propiedades> UIA_HasKeyboardFocus:= <False> UIA_IsKeyboardFocusable:= <False> UIA_IsEnabled:= <True> UIA_Culture:= <0> UIA_IsControlElement:= <True> UIA_IsContentElement:= <True> UIA_IsPassword:= <False> UIA_NativeWindowHandle:= <0> UIA_IsOffscreen:= <False> UIA_Orientation:= <0> UIA_IsRequiredForForm:= <False> UIA_IsDockPatternAvailable:= <False> UIA_IsExpandCollapsePatternAvailable:= <False> UIA_IsGridItemPatternAvailable:= <False> UIA_IsGridPatternAvailable:= <False> UIA_IsInvokePatternAvailable:= <True> UIA_IsMultipleViewPatternAvailable:= <False> UIA_IsRangeValuePatternAvailable:= <False> UIA_IsScrollPatternAvailable:= <False> UIA_IsScrollItemPatternAvailable:= <False> UIA_IsSelectionItemPatternAvailable:= <False> UIA_IsSelectionPatternAvailable:= <False> UIA_IsTablePatternAvailable:= <False> UIA_IsTableItemPatternAvailable:= <False> UIA_IsTextPatternAvailable:= <False> UIA_IsTogglePatternAvailable:= <False> UIA_IsTransformPatternAvailable:= <False> UIA_IsValuePatternAvailable:= <False> UIA_IsWindowPatternAvailable:= <False> UIA_ValueIsReadOnly:= <True> UIA_RangeValueValue:= <0> UIA_RangeValueIsReadOnly:= <True> UIA_RangeValueMinimum:= <0> UIA_RangeValueMaximum:= <0> UIA_RangeValueLargeChange:= <0> UIA_RangeValueSmallChange:= <0> UIA_ScrollHorizontalScrollPercent:= <0> UIA_ScrollHorizontalViewSize:= <100> UIA_ScrollVerticalScrollPercent:= <0> UIA_ScrollVerticalViewSize:= <100> UIA_ScrollHorizontallyScrollable:= <False> UIA_ScrollVerticallyScrollable:= <False> UIA_SelectionCanSelectMultiple:= <False> UIA_SelectionIsSelectionRequired:= <False> UIA_GridRowCount:= <0> UIA_GridColumnCount:= <0> UIA_GridItemRow:= <0> UIA_GridItemColumn:= <0> UIA_GridItemRowSpan:= <1> UIA_GridItemColumnSpan:= <1> UIA_DockDockPosition:= <5> UIA_ExpandCollapseExpandCollapseState:= <3> UIA_MultipleViewCurrentView:= <0> UIA_WindowCanMaximize:= <False> UIA_WindowCanMinimize:= <False> UIA_WindowWindowVisualState:= <0> UIA_WindowWindowInteractionState:= <0> UIA_WindowIsModal:= <False> UIA_WindowIsTopmost:= <False> UIA_SelectionItemIsSelected:= <False> UIA_TableRowOrColumnMajor:= <2> UIA_ToggleToggleState:= <2> UIA_TransformCanMove:= <False> UIA_TransformCanResize:= <False> UIA_TransformCanRotate:= <False> UIA_IsLegacyIAccessiblePatternAvailable:= <True> UIA_LegacyIAccessibleChildId:= <0> UIA_LegacyIAccessibleName:= <Propiedades> UIA_LegacyIAccessibleDescription:= <Propiedades> UIA_LegacyIAccessibleRole:= <43> UIA_LegacyIAccessibleState:= <128> UIA_LegacyIAccessibleDefaultAction:= <Press> UIA_IsDataValidForForm:= <False> UIA_ProviderDescription:= <[pid:13052,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]> UIA_IsItemContainerPatternAvailable:= <False> UIA_IsVirtualizedItemPatternAvailable:= <False> UIA_IsSynchronizedInputPatternAvailable:= <False> UIA_OptimizeForVisualContent:= <False> UIA_IsObjectModelPatternAvailable:= <False> UIA_AnnotationAnnotationTypeId:= <60000> UIA_IsAnnotationPatternAvailable:= <False> UIA_IsTextPattern2Available:= <False> UIA_StylesStyleId:= <0> UIA_StylesFillColor:= <0> UIA_StylesFillPatternColor:= <0> UIA_IsStylesPatternAvailable:= <False> UIA_IsSpreadsheetPatternAvailable:= <False> UIA_IsSpreadsheetItemPatternAvailable:= <False> UIA_Transform2CanZoom:= <False> UIA_IsTransformPattern2Available:= <False> UIA_LiveSetting:= <0> UIA_IsTextChildPatternAvailable:= <False> UIA_IsDragPatternAvailable:= <False> UIA_DragIsGrabbed:= <False> UIA_IsDropTargetPatternAvailable:= <False> UIA_Transform2ZoomLevel:= <1> UIA_Transform2ZoomMinimum:= <1> UIA_Transform2ZoomMaximum:= <1> UIA_IsTextEditPatternAvailable:= <False> UIA_IsPeripheral:= <False> UIA_IsCustomNavigationPatternAvailable:= <False> UIA_PositionInSet:= <0> UIA_SizeOfSet:= <0> UIA_Level:= <0> UIA_LandmarkType:= <0> UIA_FillType:= <0> UIA_VisualEffects:= <0> UIA_IsSelectionPattern2Available:= <False> UIA_Selection2ItemCount:= <0> UIA_HeadingLevel:= <80050>
spudw2k Posted October 2, 2019 Posted October 2, 2019 Did you stumble across this thread yet? Spoiler Things I've Made: Always On Top Tool ◊ AU History ◊ Deck of Cards ◊ HideIt ◊ ICU ◊ Icon Freezer ◊ Ipod Ejector ◊ Junos Configuration Explorer ◊ Link Downloader ◊ MD5 Folder Enumerator ◊ PassGen ◊ Ping Tool ◊ Quick NIC ◊ Read OCR ◊ RemoteIT ◊ SchTasksGui ◊ SpyCam ◊ System Scan Report Tool ◊ System UpTime ◊ Transparency Machine ◊ VMWare ESX Builder Misc Code Snippets: ADODB Example ◊ CheckHover ◊ Detect SafeMode ◊ DynEnumArray ◊ GetNetStatData ◊ HashArray ◊ IsBetweenDates ◊ Local Admins ◊ Make Choice ◊ Recursive File List ◊ Remove Sizebox Style ◊ Retrieve PNPDeviceID ◊ Retrieve SysListView32 Contents ◊ Set IE Homepage ◊ Tickle Expired Password ◊ Transpose Array Projects: Drive Space Usage GUI ◊ LEDkIT ◊ Plasma_kIt ◊ Scan Engine Builder ◊ SpeeDBurner ◊ SubnetCalc Cool Stuff: AutoItObject UDF ◊ Extract Icon From Proc ◊ GuiCtrlFontRotate ◊ Hex Edit Funcs ◊ Run binary ◊ Service_UDF
IgorCabrera Posted October 2, 2019 Author Posted October 2, 2019 Dear spudw2k, I read completely the post you suggested ( https://www.autoitscript.com/forum/topic/130371-automate-qt-and-qwidgets/page/2/) but it answered just a half part of my question, the part that refers to the changing instance of QT controls, I´ll try to refer the controls using the code: Const $handle_to_find = "paste the handle from Ranorex Spy here" For $i = 1 to 1000 $handle = ControlGetHandle("window title", "", "[CLASS:QWidget; INSTANCE:" & $i & "]") if @error = 1 Then ExitLoop if ($handle = $handle_to_find) Then ConsoleWrite("Found QWidget handle " & $handle_to_find & " at instance: " & $i & @CRLF) ExitLoop EndIf Next But, I still don’t know how to click or select the elements inside the QT control, as you can see in the attached screenshot I need to a button, RannerexSpy says it’s the element ‘Propiedades’ inside a frame with Qt5QWindowIcon instance 4, but the handle and instance belong to the frame, not to the button
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now