What is "Simple UIA Spy" ? And what is it used for ?

[kcvinu]

Hi all,

I have got a little tool named "Simple UIA Spy" from this forum. But i don't know how to use it in my script. When i press "Ctrl + W", it will show some informations like Window info tool. But when i press "Ctrl + R", it will show an error message like this(please look this image). I would like to know how to use this informations in my script. 

Edit : And what is the deffernce between Window info tool and Simple UIA Spy.

Edited March 3, 2015 by kcvinu

[SmOke_N]

Well you got it from here: '?do=embed' frameborder='0' data-embedContent>>

I believe I've read that he has documentation on how to use it in that thread... have you not found it?

[kcvinu]

Hi 

SmOke_N, You mean some links with a heading "Examples" ?. If so i didn't read it yet. Can i get a pdf tutorial or something like that for offline reading ?. Because, my AutoIt learning time is my spare time. And unfortunately, i can't access the internet in my spare time. 

[SmOke_N]

I'm sure you can copy the code into word and create your own pdf.

[kcvinu]

Now, i am saving each webpage in this particular thread into my local disk.

[junkew]

Ctrl + r is not implemented so far and will not work.

And iuiautomation without internet and google is a hard way of using iuiautomation. The uiawrappers just simplify a little from the whole microsoft iuiautomation object model.

[kcvinu]

Hi, 

junkew I have started learning _UIA_Automation. This is my first try. A code for right clicking an icon in system tray. So far so good. But i don't know how to select an item from tray icon menu

Dim $pInvoke ;~ Pattern invoke
$oTaskBar = _UIA_gettaskbar()
$oTrayItem = _UIA_getFirstObjectOfElement($oTaskbar, "name:=KeyMagic (32bit)", $treescope_subtree)


;$oInvokeP=_UIA_getpattern($oTrayItem ,$UIA_InvokePatternID)
;$oInvokeP.Invoke
_UIA_action($oTrayItem ,"rightclick")

[junkew]

and thats the nice part of the simple spy

hover to your element

press ctrl+w

and in the output you have some sample source

and if you have real luck then this source directly will work

please post the output of the simplespy as text in this thread and I can check a little more

check if example 2 works of the iuiautomation examples (could not work to language versions of OS)

check if this works "name:=KeyMagic.*"

as the framework works with regular expressions internally sometimes special regexp characters in your name pattern will screw up things

the ( and ) have special meaning in regular expression

selecting / setting focus actions: "focus", "setfocus", "activate", "switchto"

Edited March 5, 2015 by junkew

[kcvinu]

Hi 

junkew, Thanks. Let me try.

[kcvinu]

Hi 

junkew,

This is the result when i try to capture info from keymagic's tray menu. Please see this image.

I have tested the other icons in tray, but they and their menus are giving proper result in simple spy. Only keymagic's tray menu is showing this error.

Edited March 5, 2015 by kcvinu

[kcvinu]

Hi junkew, That problem solved. I ran simple spy as administrator. Then it will showing keymagics tray menu's info.

This is the info

expandcollapse popup

Mouse position is retrieved 1190-672
At least we have an element [Manage Keyboards][]

Having the following values for all properties: 
Title is: <Manage Keyboards>    Class   := <>   controltype:= <UIA_MenuItemControlTypeId>   ,<50011>    , (0000C35B)    1126;661;148;22
*** Parent Information top down ***
1: Title is: <Desktop>  Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1366;768
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"   
0: Title is: <Context>  Class   := <#32768> controltype:= <UIA_MenuControlTypeId>   ,<50009>    , (0000C359)    1123;658;154;94
"Title:=Context;controltype:=UIA_MenuControlTypeId;class:=#32768"   


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP0=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Context;controltype:=UIA_MenuControlTypeId;class:=#32768", $treescope_children)   
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("ManageKeyboards.mainwindow", "title:=Manage Keyboards;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Manage Keyboards;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Manage Keyboards>
UIA_text:= <Manage Keyboards>
UIA_regexptitle:= <Manage Keyboards>
UIA_iaccessiblechildId:= <1>
UIA_id:= <101>
UIA_handle:= <0>
UIA_RuntimeId:= <42;592212;2;-2147483646;2280;4459245;1>
UIA_BoundingRectangle:= <1126;661;148;22>
UIA_ProcessId:= <2280>
UIA_ControlType:= <50011>
UIA_LocalizedControlType:= <menu item>
UIA_Name:= <Manage Keyboards>
UIA_AccessKey:= <k>
UIA_HasKeyboardFocus:= <True>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <101>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <1>
UIA_LegacyIAccessibleName:= <Manage Keyboards>
UIA_LegacyIAccessibleRole:= <12>
UIA_LegacyIAccessibleState:= <132>
UIA_LegacyIAccessibleKeyboardShortcut:= <k>
UIA_LegacyIAccessibleDefaultAction:= <Execute>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:2280,hwnd:0x0 Annotation:Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

[junkew]

"name:=KeyMagic (32bit)" should then be "title:=Manage Keyboards;ControlType:=UIA_MenuItemControlTypeId"

[kcvinu]

Hi junkew, Can i get a format for doing this ? I mean, for selecting a menu item from system tray

step 1: Use this function (Function name)

step 2: Use this function (Function name)

...................................................

step n: Use this function (Function name)

[junkew]

$oTrayItem = _UIA_getFirstObjectOfElement($oTaskbar, "title:=Manage Keyboards;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oTrayItem ,"setfocus")

[kcvinu]

Hi 

junkew, No way. your code is not working. I don't want you to solve my problem. But i request you to explain what each function does in _UIAWrapper.au3 contains. 

Now i met with these functions---- Set_action, Get_taskbar, GetFirstObjectElement. 

Edited March 5, 2015 by kcvinu

[junkew]

You mean lines like

GetFirstObjectElement = Gets the first object within the treescope of the element
setfocus = Sets the focus to the element thats requestes

• Example 2 is doing what you asked for (at least on my system)

 

So not sure then what your question on documentation is

• And what is the deffernce between Window info tool and Simple UIA Spy: IUIAutomation can catch a little more elements, AU3Inf is finished whereas simplespy is not (although I do not have much time to enhance look and feeld)

 

Automating elements is allways a pain in the ... as control hierarchy is changing under the tips of your fingers

Approach 1

step 1 hover the mouse over the taskbar over the button "show hidden items" with simplespy and press ctrl+w (put output in a text document)

step 2 click on the hidden item buttons

step 3 hover over the new window that appears and press ctrl+w (put output in a text document)

step 4 hover over the button you want to click and press ctrl+w (put output in a text document)

If you carefully analyse the hierarchies you see its not a logical way how z-orders are there (the window is child of desktop and not of taskbar)

Approach 2

Download windows SDK with inspect.exe which will show you all the time the hierarchy of all elements (however I think you can not see it for this window as soon as you focus to inspect.exe the popup with your button dissapears)

Approach 3

Just as in example 2 dump the information to a log.txt (this is a lot of trial and error analysing everytime the actual hierarchy of the controls)

[kcvinu]

Hi 

junkew, Thanks for your reply. I have read your Example 2. It teaches a lesson about finding start menu and clicking it. So i used the same function for finding a tray icon and right click on it. That was a success. But then selecting a menu item from tray is not included in that example. That's where i am stucking. Anyhow, let me play with it. May be someday it will come to me. 

[kcvinu]

Hi, 

Atlast i found that KeyMagic 1.4 is not supporting any kind of automation. Because, when i right click it's tray icon, it's tray menu will come. But none of my script commands will work right after that. If i need to manually exit my script, i need to click either in SciTE menu or Script icon in tray. But i can't click any where (only left clicks). My mouse clicks are then useless. But if i right click anywhere, everything will be normal. So what is happening after i automated this keymagic tray icon ?. 

About the program.

Keymagic 1.4. It is an input method editor. It helps me to type my native language in PC. I think it uses SetWindowsHookEx API.