Set text on a control using IUIAutomation.

[ngskicker]

I don't know if this is a correct place to ask this, so forgive me if this is the wrong place.

I need to set text in a text field, and then press enter, this is a text field on "Microsoft Access Database Project", this control is not visible on AutoItInfo, below is what I get from simplespy

expandcollapse popup

Mouse position is retrieved 433-228
At least we have an element [20.40.22][OKttbx]

Having the following values for all properties:
Title is: <20.40.22>    Class := <OKttbx>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    366;221;151;15
*** Parent Information top down ***
4: Title is: <Desktop>    Class := <#32769>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;0;1366;768
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"    
3: Title is: <Registrasi Px RSMLA>    Class := <OMain>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -8;-8;1382;744
"Title:=Registrasi Px RSMLA;controltype:=UIA_WindowControlTypeId;class:=OMain"    
2: Title is: <Workspace>    Class := <MDIClient>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;74;1366;654
"Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient"    
1: Title is: <Appointment Form>    Class := <OForm>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    259;124;847;537
"Title:=Appointment Form;controltype:=UIA_WindowControlTypeId;class:=OForm"    
0: Title is: <>    Class := <OFormSub>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    262;197;824;343
"Title:=;controltype:=UIA_PaneControlTypeId;class:=OFormSub"    


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Registrasi Px RSMLA;controltype:=UIA_WindowControlTypeId;class:=OMain", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Appointment Form;controltype:=UIA_WindowControlTypeId;class:=OForm", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=OFormSub", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar("20.40.22.mainwindow","title:=20.40.22;classname:=OKttbx")
_UIA_action("20.40.22.mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_title:= <20.40.22>
UIA_text:= <20.40.22>
UIA_regexptitle:= <20.40.22>
UIA_class:= <OKttbx>
UIA_regexpclass:= <OKttbx>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <1311620>
UIA_RuntimeId:= <42;1311620>
UIA_BoundingRectangle:= <366;221;151;15>
UIA_ProcessId:= <1404>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <pane>
UIA_Name:= <20.40.22>
UIA_HasKeyboardFocus:= <True>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <OKttbx>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <1311620>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <20.40.22>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048580>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:5708,hwnd:0x140384 Main:Nested [pid:1404,hwnd:0x140384 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

Edited December 18, 2014 by ngskicker

[computergroove]

'?do=embed' frameborder='0' data-embedContent>>

Maybe _AccessTableEdit() will work

[ngskicker]

'?do=embed' frameborder='0' data-embedContent>>

Maybe _AccessTableEdit() will work

This will not work, this is not general access database (.mdb), this is "Access Database Project"  (.adp) this type of file only exist until Access 2003, 

[junkew]

you have to provide a little more info. If the proper area is highlighted by the simplespy you normally can reach it.

whats the result of the code that simplespy gives you?

;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Registrasi Px RSMLA;controltype:=UIA_WindowControlTypeId;class:=OMain", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Appointment Form;controltype:=UIA_WindowControlTypeId;class:=OForm", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=OFormSub", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar("20.40.22.mainwindow","title:=20.40.22;classname:=OKttbx")
_UIA_action("20.40.22.mainwindow","setfocus")

if above works change last line to

_UIA_action("20.40.22.mainwindow","setvalue", "123")

alternatives

"setValue using keys","setValue using clipboard","sendkeys", "enterstring", "type", "typetext"

could be this does not behave properly in framework

"20.40.22.mainwindow"

make it something like below by replacing dots in first part with underscores

"20_40_22.mainwindow"

[ngskicker]

you have to provide a little more info. If the proper area is highlighted by the simplespy you normally can reach it.

whats the result of the code that simplespy gives you?

;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Registrasi Px RSMLA;controltype:=UIA_WindowControlTypeId;class:=OMain", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Appointment Form;controltype:=UIA_WindowControlTypeId;class:=OForm", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=OFormSub", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar("20.40.22.mainwindow","title:=20.40.22;classname:=OKttbx")
_UIA_action("20.40.22.mainwindow","setfocus")

if above works change last line to

_UIA_action("20.40.22.mainwindow","setvalue", "123")

alternatives

"setValue using keys","setValue using clipboard","sendkeys", "enterstring", "type", "typetext"

could be this does not behave properly in framework

"20.40.22.mainwindow"

make it something like below by replacing dots in first part with underscores

"20_40_22.mainwindow"

unfortunately the code above is not working, I can confirm the textbox is highlighted when I press Ctrl+W, but not when I run the script above, the code _UIA_Action($oP0,"setfocus") still works, but code below it is not, by the way "20.40.22" is value of the textbox, this value is not always same.

[LarsJ]

It may be difficult to identify the proper control. I would recommend you to use Inspect.exe from Windows SDK to verify that it's the right control. Inspect.exe also shows how the control is placed in the hierarchy of windows and container controls. When you have identified the proper control, you can use the UI Automation framework to set the text.

[junkew]

Simple spy proves it can work. Most likely you have to use instance property. The xxxxxx.mainwindow part should xxxxxxx not contain dots else it will not work. So 204022.mainwindow you could try. And try to remove title:=20.40.22 as classname seems to be unique or try classname with instance:=nn with nn being 1 2 3 etc

If you say it works up till setfocus you should see box highlghting but then setvalue is not workng due to not iuia compatible and you should try the other options given

Edited December 22, 2014 by junkew

[junkew]

"setValue using keys","setValue using clipboard","sendkeys", "enterstring", "type", "typetext"

just to make sure this is what i mean

_UIA_setVar("204022.mainwindow","classname:=OKttbx;instance:=1")
_UIA_action("204022.mainwindow","setfocus")
_UIA_action("204022.mainwindow","setvalue using keys","hello world")
_UIA_action("204022.mainwindow","setvalue using clipboard","hello world")

all those variations of setting values use slightly different ways of dealing with the not so standard textboxes

setvalue uses iuiautomation which unfortunately is not supported by all controls (you should complain to your supplier if possible)

Edited December 22, 2014 by junkew

[ngskicker]

"setValue using keys","setValue using clipboard","sendkeys", "enterstring", "type", "typetext"

just to make sure this is what i mean

_UIA_setVar("204022.mainwindow","classname:=OKttbx;instance:=1")
_UIA_action("204022.mainwindow","setfocus")
_UIA_action("204022.mainwindow","setvalue using keys","hello world")
_UIA_action("204022.mainwindow","setvalue using clipboard","hello world")

all those variations of setting values use slightly different ways of dealing with the not so standard textboxes

setvalue uses iuiautomation which unfortunately is not supported by all controls (you should complain to your supplier if possible)

Hi junkew, thanks, I'll try it onthe next hour, 

but "20.40.22" is control value, and will always have different value, and sometimes empty

[junkew]

"204022.mainwindow" with quotes around it is just a reference name, something like a variable and not related to the value of the box.

It was just the value at the moment the spy looked at it.

you just as wel can use

"abc.mainwindow"

using it in a property like this

title:=20.40.22

means look for a control with contentvalue/title to be 20.40.22

as you say it can be empty or unknown values its not smart to find based on title property and you have to fall back to other properties to uniquely identify your control

Edited December 23, 2014 by junkew

[LarsJ]

junkew, I'm pretty sure, that this pane control in first post is not the proper control. If ngskicker manually can edit the control with the keyboard (if not, he will not be able to set the text automatically with UI Automation), then it's not a pane control. Then it's an edit control.

What's going on is that the pane control is a container for the edit control. (The edit control is a child control of the pane control.) Because the edit control is below the pane control, Simple Spy will return the pane control. Simple Spy returns the first control it sees at the position. With the current implementation of Simple Spy I don't think it'll be able to return the edit control.

This limitation is a main issue for the Simple Spy script. And it's the cause of many of the posts about UI Automation. It's not returning the control, that the users think it's returning. E.g. it's returning a pane control, and the user think it's an edit control.

Use Inspect.exe to get the proper edit control below the pane control.

Merry Christmas by the way.

[junkew]

@LarsJ: Thx. Will enhance simple spy in future version to return information of childcontrols and if its less then 5 childs display base information on the childs. Wondering how inspect.exe retrieves the right control or only can get there with a treewalker. seems uipane does not have the child controls in MS Access

Just adding this in simplespy below

_UIA_DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])

;~  Just walk thru all childs
;~  at least 1 assumed (assuming we are not spying the desktop)
    $i=0
    local $oChildHandle
    local $pChildHandle

    local $oParentHandle
    local $pParentHandle
    $oTW.getparentelement($oUIElement,$pParentHandle)
    $oParentHandle=objcreateinterface($pparentHandle,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    $oTW.GetFirstChildElement($oParenthandle,$pChildHandle)
    $oChildHandle=objcreateinterface($pChildHandle,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If IsObj($oChildHandle) = 0 Then
        msgbox(1,"No childs", "No childs below this element",10)
    Else

        while (IsObj($oChildHandle)=1)
            consolewrite("title:=" & _UIA_getPropertyValue($oChildHandle,$UIA_NamePropertyId) & ";classname:=" & _UIA_getPropertyValue($oUIElement,$uia_classnamepropertyid) & @CRLF)
            $oTW.GetNextSiblingElement($oChildHandle,$pChildHandle)
            $oChildHandle=objcreateinterface($pChildHandle,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)

        wend
    EndIf 

 

Edited December 24, 2014 by junkew

[LarsJ]

The width/height of the pane control in first post is 151/15. This probably means that the pane control only contains the edit control. And it's likely that the edit control has the exact same size.

Child controls. I think that you only have to care about child controls in case that the current control is a pane control.

Generally there seems to be some problems with pane controls. E.g. this issue at stackoverflow. Note that Inspect.exe can find the controls.

Simple Spy. I would like to see the list of parent controls from the current control to desktop printed out in opposite order. So that the first parent is immediately below the current control.

[ngskicker]

Hi junkew, I tried everything you mentioned, but I still cant get it to work, if I can I want to send the program, but unfortunately, the program need a database that can't be sent.

[junkew]

@ngskicker: Best to see if it can work is indeed install inspect.exe frow windows SDK as that way you can navigate thru the hierarchy and see if your control is really recognized as type Edit (and not type pane)

I partly can reproduce simplespy not recognizing controls on an ms access form but need more time to fix simplespy icm with current framework IUIA I am working on.