Using UI Automation Code in AutoIt

[LarsJ]

You will need to find out what type of automation is supported by your application. Classic (AutoIt) automation, MSAA automation (Inspect.exe, dropdown control in upper left corner) or UI Automation (Inspect.exe or UIASpy). For both MSAA and UI Automation code, the rule of thumb is that it's only possible to automate a task that can be performed manually by an end user.

[mLipok]

I'm just looking for a small .NET application that is publicly available to make tests available to everyone.

[LarsJ]

You cannot know in advance which automation options will work. If you want to make something that is generally applicable, then you will need to make some small tests for all 3 options. But I have no immediate ideas about possible test applications.

[mLipok]

Here is better repro:

expandcollapse popup

#AutoIt3Wrapper_UseX64=y
#RequireAdmin
;~ https://www.autoitscript.com/forum/topic/197080-using-ui-automation-code-in-autoit/?do=findComment&comment=1513436

Global $_UIA_SMART_AUTOMATION, $_UIA_SMART_DESKTOP
#include <GUIConstantsEx.au3>
#include "z:\!!!_SVN_AU3\UDF_Forum\Other_Members\UIA_Constants.au3"

Example()

Func Example()
    ; Create a GUI with various controls.
    Local $hGUI = GUICreate("UIA HWND Example")
    Local $idOK = GUICtrlCreateButton("OK", 310, 370, 85, 25)

    ; Display the GUI.
    GUISetState(@SW_SHOW, $hGUI)

    _UIA_CreateProperty_issue_testing()

    ; Loop until the user exits.
    While 1


        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE, $idOK
                ExitLoop

        EndSwitch
    WEnd

    ; Delete the previous GUI and all controls.
    GUIDelete($hGUI)
EndFunc   ;==>Example

Func _UIA_CreateProperty_issue_testing()

    _Log("! WinWait('UIA HWND Example') = " & WinWait('UIA HWND Example'))
    _UIASimple_GetWindowByTitle('UIA HWND Example', '')


EndFunc   ;==>_UIA_CreateProperty_issue_testing

Func _UIASimple_GetWindowByTitle($sTitle, $sText = '')
    #forceref $sText
    Local $sHWND = 0

    ConsoleWrite("> " & @ScriptLineNumber & ' $sHWND=' & $sHWND &' $sTitle=' & $sTitle &' =' &' =' & '' &'' &@CRLF)

    Local $oUIA = _UIASimple_InitAutomation()
    Local $oDesktop = _UIASimple_GetDesktop()

    Local $pCondition1
    $oUIA.CreatePropertyCondition($UIA_NamePropertyId, $sTitle, $pCondition1)
;~  $oUIA.CreatePropertyCondition($UIA_NativeWindowHandlePropertyId, $sHWND, $pCondition1)
    Local $s_Info = ' ::: Title=' & $sTitle & (($pCondition1) ? ('') : (" ::: $pCondition1 ERR ::: "))
    _Log(@ScriptLineNumber & $s_Info)
;~  Local $s_Info = ' HWND = ' & $sHWND & ' ::: Title=' & $sTitle & (($pCondition1) ? ('') : (" ::: $pCondition1 ERR ::: "))

    If Not $pCondition1 Then Return

    Local $pWindow
    $oDesktop.FindFirst($TreeScope_Descendants, $pCondition1, $pWindow)
    _Log(@ScriptLineNumber & " $pWindow= " & $pWindow)
    Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oWindow) Then Return _Log(@ScriptLineNumber & " $oWindow ERR - Title= " & $sTitle)
    _Log(@ScriptLineNumber & " $oWindow OK")

    Local $hWnd2
    $hWnd2 = $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, False)
    Local $s_Title2 = $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, True)
;~  $hWnd2 = $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, False)
    _Log(@ScriptLineNumber & ' $hWnd2 = ' & $hWnd2 & ' $s_Title2 = ' & $s_Title2)

    Return $oWindow
EndFunc   ;==>_UIASimple_GetWindowByTitle

Func _Log($sData, $iERR = @error, $iEXT = @extended)
    ConsoleWrite($sData & @CRLF)
    Return SetError($iERR, $iEXT)
EndFunc   ;==>_Log

Func _UIASimple_InitAutomation()
    If IsObj($_UIA_SMART_AUTOMATION) Then Return $_UIA_SMART_AUTOMATION

    ; Create UI Automation object
    Local $oUIA = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
    If Not IsObj($oUIA) Then Return _Log(@ScriptLineNumber & " $oUIA ERR")
    _Log(@ScriptLineNumber & " $oUIA OK")

    $_UIA_SMART_AUTOMATION = $oUIA
    Return $_UIA_SMART_AUTOMATION
EndFunc   ;==>_UIASimple_InitAutomation

Func _UIASimple_GetDesktop()
    If IsObj($_UIA_SMART_DESKTOP) Then Return $_UIA_SMART_DESKTOP
    Local $oUIA = _UIASimple_InitAutomation()

    ; Get Desktop element
    Local $pDesktop
    $oUIA.GetRootElement($pDesktop)
    Local $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oDesktop) Then Return _Log(@ScriptLineNumber & " $oDesktop ERR")
    _Log(@ScriptLineNumber & " $oDesktop OK")

    $_UIA_SMART_DESKTOP = $oDesktop
    Return $_UIA_SMART_DESKTOP

EndFunc   ;==>_UIASimple_GetDesktop

Here is screenshot:

As you can see UIASpy properly sees the $UIA_NativeWindowHandlePropertyId value.

And my question related to this screenshot:

Why I can not do the same (to get $UIA_NativeWindowHandlePropertyId value) with my repro code ?

 

Edited February 16, 2023 by mLipok

[InnI]

30 minutes ago, mLipok said:

to get $UIA_NativeWindowHandlePropertyId value

;~     Local $hWnd2
;~     $hWnd2 = $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, False)
;~     Local $s_Title2 = $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, True)
    Local $hWnd2, $s_Title2
    $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd2)
    $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, $s_Title2)

 

[InnI]

AutoIt can find any top-level window. No need to search from desktop. Just use method ElementFromHandle. This method can be used for any controls that have a native handle (found by AutoIt).

expandcollapse popup

#AutoIt3Wrapper_UseX64=y

Global $_UIA_SMART_AUTOMATION

#include <GUIConstantsEx.au3>
#include "UIA_Constants.au3"

Example()

Func Example()
    Local $hGUI = GUICreate("UIA HWND Example")
    Local $idOK = GUICtrlCreateButton("OK", 310, 370, 85, 25)
    GUISetState(@SW_SHOW, $hGUI)
    ; ---------------------------------
    ; main window
    $hWnd = WinWait("UIA HWND Example")
    _Log("! hWnd = " & $hWnd)
    $oWindow = _UIASimple_ElementFromHandle($hWnd)
    Local $hWnd2, $s_Title2
    $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd2)
    $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, $s_Title2)
    _Log("! hWnd2 = 0x" & Hex($hWnd2) & ", $s_Title2 = " & $s_Title2)
    _Log("----------------")

    ; button OK
    $hCtrl = ControlGetHandle("UIA HWND Example", "", "Button1")
    _Log("! hCtrl = " & $hCtrl)
    $oBtn = _UIASimple_ElementFromHandle($hCtrl)
    Local $hCtrl2, $sText
    $oBtn.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hCtrl2)
    $oBtn.GetCurrentPropertyValue($UIA_NamePropertyId, $sText)
    _Log("! hCtrl2 = 0x" & Hex($hCtrl2) & ", $sText = " & $sText)
    ; ---------------------------------
    While 1
        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE, $idOK
                ExitLoop
        EndSwitch
    WEnd
    GUIDelete($hGUI)
EndFunc   ;==>Example

Func _UIASimple_ElementFromHandle($hHandle)
  If Not IsHWnd($hHandle) Then Return SetError(1, 0, 0)
  Local $pElement, $oElement, $oUIA = _UIASimple_InitAutomation()
  $oUIA.ElementFromHandle($hHandle, $pElement)
  $oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
  $pElement = 0
  If Not IsObj($oElement) Then Return SetError(2, 0, 0)
  Return $oElement
EndFunc   ;==>_UIASimple_ElementFromHandle

Func _UIASimple_InitAutomation()
    If IsObj($_UIA_SMART_AUTOMATION) Then Return $_UIA_SMART_AUTOMATION
    Local $oUIA = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
    If Not IsObj($oUIA) Then Return _Log(@ScriptLineNumber & " $oUIA ERR")
    $_UIA_SMART_AUTOMATION = $oUIA
    Return $_UIA_SMART_AUTOMATION
EndFunc   ;==>_UIASimple_InitAutomation

Func _Log($sData, $iERR = @error, $iEXT = @extended)
    ConsoleWrite($sData & @CRLF)
    Return SetError($iERR, $iEXT)
EndFunc   ;==>_Log

 

Edited February 16, 2023 by InnI

[mLipok]

@InnI thanks for:

Func _UIASimple_ElementFromHandle($hHandle)
  If Not IsHWnd($hHandle) Then Return SetError(1, 0, 0)
  Local $pElement, $oElement, $oUIA = _UIASimple_InitAutomation()
  $oUIA.ElementFromHandle($hHandle, $pElement)
  $oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
  $pElement = 0
  If Not IsObj($oElement) Then Return SetError(2, 0, 0)
  Return $oElement
EndFunc   ;==>_UIASimple_ElementFromHandle

And thanks to your suggestion about:

Local $hWnd2, $s_Title2
    $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd2)
    $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, $s_Title2)
    _Log(@ScriptLineNumber & ' $hWnd2 = ' & $hWnd2 & ' $s_Title2 = ' & $s_Title2)

 

This directs me to solutions the main problem.

Becuase I notice that:
 

$oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd2)

returns value in DEC

but:
 

WinWait('UIA HWND Example')

returns value in HEX

 

So I'm working on finall solution.

 

[mLipok]

The final solution:

expandcollapse popup

#AutoIt3Wrapper_UseX64=y
;~ #AutoIt3Wrapper_UseX64=n
#RequireAdmin
;~ https://www.autoitscript.com/forum/topic/197080-using-ui-automation-code-in-autoit/?do=findComment&comment=1513436

Global $_UIA_SMART_AUTOMATION, $_UIA_SMART_DESKTOP
#include <AutoItConstants.au3>
#include <GUIConstantsEx.au3>
#include "z:\!!!_SVN_AU3\UDF_Forum\Other_Members\UIA_Constants.au3"

Example()

Func Example()
    ; Create a GUI with various controls.
    Local $hGUI = GUICreate("UIA HWND Example")
    Local $idOK = GUICtrlCreateButton("OK", 310, 370, 85, 25)

    ; Display the GUI.
    GUISetState(@SW_SHOW, $hGUI)

    _UIA_CreateProperty_issue_testing()

    ; Loop until the user exits.
    While 1


        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE, $idOK
                ExitLoop

        EndSwitch
    WEnd

    ; Delete the previous GUI and all controls.
    GUIDelete($hGUI)
EndFunc   ;==>Example

Func _UIA_CreateProperty_issue_testing()

    Local $hWND_HEX = WinWait('UIA HWND Example')
    Local $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2), $NUMBER_32BIT)
    _Log("! WinWait('UIA HWND Example') = " & $hWND_HEX)
    _Log("! WinWait('UIA HWND Example') = " & $hWND_DEC)

    ConsoleWrite("- FIRST TEST - Title" & @CRLF)
    _UIASimple_GetWindowByTitle('UIA HWND Example', '')

    ConsoleWrite("- SECOND TEST - HWND" & @CRLF)
    _UIASimple_GetWindowByTitle($hWND_HEX, '')

    ConsoleWrite("- THIRD TEST - HWND" & @CRLF)
    _UIASimple_ElementFromHandle($hWND_HEX)

    ConsoleWrite("- TEST END" & @CRLF)

EndFunc   ;==>_UIA_CreateProperty_issue_testing

Func _UIASimple_GetWindowByTitle($sTitle, $sText = '')
    #forceref $sText ; Not supported yet
    Local $hWND_HEX = 0
    Local $hWND_DEC = 0

    If IsHWnd($sTitle) Then
        $hWND_HEX = $sTitle
        $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2), $NUMBER_32BIT)
        $sTitle = WinGetTitle($sTitle)
    EndIf

    Local $oUIA = _UIASimple_InitAutomation()
    Local $oDesktop = _UIASimple_GetDesktop()

    Local $pCondition1
    If $hWND_DEC Then
        _Log(@ScriptLineNumber & ' VargetType($hWND_DEC) = ' & VarGetType($hWND_DEC))
        $oUIA.CreatePropertyCondition($UIA_NativeWindowHandlePropertyId, $hWND_DEC, $pCondition1)
    Else
        $oUIA.CreatePropertyCondition($UIA_NamePropertyId, $sTitle, $pCondition1)
    EndIf

    Local $s_Info = ' ::: $hWND_DEC=' & $hWND_DEC & ' ::: Title=' & $sTitle & (($pCondition1) ? ('') : (" ::: $pCondition1 ERR ::: "))
    _Log(@ScriptLineNumber & $s_Info)

    If Not $pCondition1 Then Return

    Local $pWindow
    $oDesktop.FindFirst($TreeScope_Descendants, $pCondition1, $pWindow)
    _Log(@ScriptLineNumber & " $pWindow= " & $pWindow)
    Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oWindow) Then Return _Log(@ScriptLineNumber & " $oWindow ERR - Title= " & $sTitle)
    _Log(@ScriptLineNumber & " $oWindow OK")

    Local $hWnd_Native, $s_Title_FromUIA
    $oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd_Native)
    $oWindow.GetCurrentPropertyValue($UIA_NamePropertyId, $s_Title_FromUIA)
    _Log(@ScriptLineNumber & ' $hWnd_Native = ' & $hWnd_Native & ' $s_Title_FromUIA = ' & $s_Title_FromUIA)
    _Log(@ScriptLineNumber & ' VargetType($hWnd_Native) = ' & VarGetType($hWnd_Native))

    Return $oWindow
EndFunc   ;==>_UIASimple_GetWindowByTitle

Func _UIASimple_InitAutomation()
    If IsObj($_UIA_SMART_AUTOMATION) Then Return $_UIA_SMART_AUTOMATION

    ; Create UI Automation object
    Local $oUIA = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
    If Not IsObj($oUIA) Then Return _Log(@ScriptLineNumber & " $oUIA ERR")
    _Log(@ScriptLineNumber & " $oUIA OK")

    $_UIA_SMART_AUTOMATION = $oUIA
    Return $_UIA_SMART_AUTOMATION
EndFunc   ;==>_UIASimple_InitAutomation

Func _UIASimple_GetDesktop()
    If IsObj($_UIA_SMART_DESKTOP) Then Return $_UIA_SMART_DESKTOP
    Local $oUIA = _UIASimple_InitAutomation()

    ; Get Desktop element
    Local $pDesktop
    $oUIA.GetRootElement($pDesktop)
    Local $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oDesktop) Then Return _Log(@ScriptLineNumber & " $oDesktop ERR")
    _Log(@ScriptLineNumber & " $oDesktop OK")

    $_UIA_SMART_DESKTOP = $oDesktop
    Return $_UIA_SMART_DESKTOP

EndFunc   ;==>_UIASimple_GetDesktop

; #FUNCTION# ====================================================================================================================
; Name ..........: _UIASimple_ElementFromHandle
; Description ...:
; Syntax ........: _UIASimple_ElementFromHandle($hHandle)
; Parameters ....: $hHandle             - a handle value.
; Return values .: None
; Author ........: InnI
; Modified ......:
; Remarks .......:
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _UIASimple_ElementFromHandle($hHandle)
    If Not IsHWnd($hHandle) Then Return SetError(1, 0, 0)
    Local $pElement, $oElement, $oUIA = _UIASimple_InitAutomation()
    $oUIA.ElementFromHandle($hHandle, $pElement)
    $oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    $pElement = 0
    If Not IsObj($oElement) Then Return SetError(2, 0, 0)
    Local $hWnd_Native, $s_Title_FromUIA
    $oElement.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd_Native)
    $oElement.GetCurrentPropertyValue($UIA_NamePropertyId, $s_Title_FromUIA)
    _Log(@ScriptLineNumber & ' $oElement OK ::: $hWnd_Native = ' & $hWnd_Native & ' $s_Title_FromUIA = ' & $s_Title_FromUIA)
    Return $oElement
EndFunc   ;==>_UIASimple_ElementFromHandle

Func _Log($sData, $iERR = @error, $iEXT = @extended)
    ConsoleWrite($sData & @CRLF)
    Return SetError($iERR, $iEXT)
EndFunc   ;==>_Log

 

btw. take a note that 

 

$oWindow.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd2)

returns a value in DEC, it seems to always be a 32 bit DEC value

but:
 

 WinWait('UIA HWND Example')

returns value in HEX with bitness corelated to:

#AutoIt3Wrapper_UseX64=y
;~ #AutoIt3Wrapper_UseX64=n

Edit 1:
Thus this following line was needed:

Local $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2), $NUMBER_32BIT)

Edit 2:
Results:

Quote

! WinWait('UIA HWND Example') = 0x0000000000320BFA
! WinWait('UIA HWND Example') = 3279866
- FIRST TEST - Title
107 $oUIA OK
122 $oDesktop OK
81 ::: $hWND_DEC=0 ::: Title=UIA HWND Example
87 $pWindow= 1742066804512
90 $oWindow OK
95 $hWnd_Native = 3279866 $s_Title_FromUIA = UIA HWND Example
96 VargetType($hWnd_Native) = Int32
- SECOND TEST - HWND
74 VargetType($hWND_DEC) = Int32
81 ::: $hWND_DEC=3279866 ::: Title=UIA HWND Example
87 $pWindow= 1742066804512
90 $oWindow OK
95 $hWnd_Native = 3279866 $s_Title_FromUIA = UIA HWND Example
96 VargetType($hWnd_Native) = Int32
- THIRD TEST - HWND
152 $oElement OK ::: $hWnd_Native = 3279866 $s_Title_FromUIA = UIA HWND Example
- TEST END
 

 

Edited February 16, 2023 by mLipok
edit 1 , 2 + typos

[mLipok]

@LarsJ What you think about to change UIASpy to  show $UIA_NativeWindowHandlePropertyId value in correct bitness and togther HEX + DEC ?

[InnI]

22 minutes ago, mLipok said:

Thus this following line was needed

HEX and DEC are just a representation of a number. Their values are the same.

[mLipok]

6 minutes ago, InnI said:

HEX and DEC are just a representation of a number. Their values are the same.

so Instead:

Local $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2), $NUMBER_32BIT)

try to use:

Local $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2))

and switch between:

#AutoIt3Wrapper_UseX64=y
;~ #AutoIt3Wrapper_UseX64=n

What results you get ?

EDIT: 
oops I mean try here:

If IsHWnd($sTitle) Then
        $hWND_HEX = $sTitle
        $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2), $NUMBER_32BIT)
        $sTitle = WinGetTitle($sTitle)
    EndIf

to change to:

If IsHWnd($sTitle) Then
        $hWND_HEX = $sTitle
        $hWND_DEC = Dec(StringTrimLeft($hWND_HEX, 2))
        $sTitle = WinGetTitle($sTitle)
    EndIf

 

Edited February 16, 2023 by mLipok

[InnI]

1 hour ago, mLipok said:

it seems to always be a 32 bit

Looks like you're right.

https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-automation-element-propids

UIA_NativeWindowHandlePropertyId 30020

Identifies the NativeWindowHandle property, which is an integer that represents the handle (HWND) of the automation element window, if it exists; otherwise, this property is 0. Variant type: VT_I4 Default value: 0

And...

; Return values .: UIA Element (object)
; Author ........: InnI
; Modified ......: mLipok

😉

[demon964]

 

Hi. I need add email and password to this textbox, but cant find Autoit window info

[demon964]

hi, how i can use IUIAutomation for login microsoft store?  or simple paste text to microsoft store textbox

[Danyfirex]

@demon964

You can do it like this:

 

expandcollapse popup

#include "UIA_Constants.au3"


_Test()

Func _Test()
    ; --- Find window/control ---

    ConsoleWrite("--- Find window/control ---" & @CRLF)
    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("$oUIAutomation ERR" & @CRLF)
    ConsoleWrite("$oUIAutomation OK" & @CRLF)

    ; Get Desktop element
    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement($pDesktop)
    $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oDesktop) Then Return ConsoleWrite("$oDesktop ERR" & @CRLF)
    ConsoleWrite("$oDesktop OK" & @CRLF)

    Local $pCondition1
    $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, "ApplicationFrameWindow", $pCondition1)
    If Not $pCondition1 Then Return ConsoleWrite("$pCondition1 ERR" & @CRLF)
    ConsoleWrite("$pCondition1 OK" & @CRLF)

    Local $pWindow1, $oWindow1
    $oDesktop.FindFirst($TreeScope_Children, $pCondition1, $pWindow1)
    $oWindow1 = ObjCreateInterface($pWindow1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oWindow1) Then Return ConsoleWrite("$oWindow1 ERR" & @CRLF)
    ConsoleWrite("$oWindow1 OK" & @CRLF)

    Local $pCondition0
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Enter your email, phone, or Skype.", $pCondition0) ;change this for your Russian Edit Text
    If Not $pCondition0 Then Return ConsoleWrite("$pCondition0 ERR" & @CRLF)
    ConsoleWrite("$pCondition0 OK" & @CRLF)

    Local $pEdit1, $oEdit1
    $oWindow1.FindFirst($TreeScope_Descendants, $pCondition0, $pEdit1)
    $oEdit1 = ObjCreateInterface($pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oEdit1) Then Return ConsoleWrite("$oEdit1 ERR" & @CRLF)
    ConsoleWrite("$oEdit1 OK" & @CRLF)


    Local $pLegacyIAccessiblePattern1, $oLegacyIAccessiblePattern1
    $oEdit1 .GetCurrentPattern($UIA_LegacyIAccessiblePatternId, $pLegacyIAccessiblePattern1)
    $oLegacyIAccessiblePattern1 = ObjCreateInterface($pLegacyIAccessiblePattern1, $sIID_IUIAutomationLegacyIAccessiblePattern, $dtag_IUIAutomationLegacyIAccessiblePattern)
    If Not IsObj($oLegacyIAccessiblePattern1) Then Return ConsoleWrite("$oLegacyIAccessiblePattern1 ERR" & @CRLF)
    ConsoleWrite("$oLegacyIAccessiblePattern1 OK" & @CRLF)

    Local $iStr = "Danyfirex@autoit.com"

    $oLegacyIAccessiblePattern1.SetValue($iStr)

EndFunc   ;==>_Test

 

Saludos

[demon964]

@Danyfirex you compiled code?  automatically added your mail adress to microsoft store?    mine not worked

[Danyfirex]

that's the first step just add the email you need to do/add some code to full log in.  I only teach you how to catch the fish, but if you only want the fish you have to pay for it. 😅

 

Saludos

[Melba23]

demon964,

You already have one thread helping you on this, so why post in another? Let us stick to this one.

M23

[SOLVE-SMART]

Hi @demon964 ,

do you plan to automated more then the login (account confirmation)?
In case it's only this window/popup/modal, you could save your time by simply

• find the window
• focus the window
• navigate through TABs
• send your text (inputs)
• use SPACE or ENTER to confirm the button(s)

.. done.

Of course this wouldn't be a nice approach, but maybe a simple one in case nothing more is to do 🤔 ?!

Best regards
Sven

[jchd]

This is the first time I actually need to use UI Automation, but I can't reach any climax.

This is the UIASpy of the Firefox window I need to enter text in several Edit controls.

My code so far:

expandcollapse popup

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "Includes\UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_SafeArray.au3" ; Can be copied from UIASpy Includes folder
;#include "UIA_Variant.au3" ; Can be copied from UIASpy Includes folder

Opt("MustDeclareVars", 1)

Example()

Func Example()

    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("$oUIAutomation ERR" & @CRLF)
    ConsoleWrite("$oUIAutomation OK" & @CRLF)

    ; Get Desktop element
    Local $pDesktop, $oDesktop
    $oUIAutomation.GetRootElement($pDesktop)
    $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oDesktop) Then Return ConsoleWrite("$oDesktop ERR" & @CRLF)
    ConsoleWrite("$oDesktop OK" & @CRLF)

    ; --- Find window ---

    ConsoleWrite("--- Find window ---" & @CRLF)

    Local $pCondition
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Votre colis - Colissimo en ligne - Boutique La Poste pour les professionnels — Mozilla Firefox", $pCondition)
    If Not $pCondition Then Return ConsoleWrite("$pCondition ERR" & @CRLF)
    ConsoleWrite("$pCondition OK" & @CRLF)

    Local $pWindow, $oWindow
    $oDesktop.FindFirst($TreeScope_Children, $pCondition, $pWindow)
    $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oWindow) Then Return ConsoleWrite("$oWindow ERR" & @CRLF)
    ConsoleWrite("$oWindow OK" & @CRLF)

    ; --- Find document ---

    ConsoleWrite("--- Find document ---" & @CRLF)

    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_DocumentControlTypeId, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("$pCondition ERR" & @CRLF)
    ConsoleWrite("$pCondition OK" & @CRLF)

    Local $pDocument, $oDocument
    $oWindow.FindFirst($TreeScope_Descendants, $pCondition, $pDocument)
    $oDocument = ObjCreateInterface($pDocument, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oDocument) Then Return ConsoleWrite("$oDocument ERR" & @CRLF)
    ConsoleWrite("$oDocument OK" & @CRLF)

    ConsoleWrite("--- Find control ---" & @CRLF)

    $oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, "addressFormDestPart.lastName", $pCondition)
    If Not $pCondition Then Return ConsoleWrite("$pCondition ERR" & @CRLF)
    ConsoleWrite("$pCondition OK" & @CRLF)

    Local $pEdit, $oEdit
    $oDocument.FindFirst($TreeScope_Descendants, $pCondition, $pEdit)
    $oEdit = ObjCreateInterface($pEdit, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oEdit) Then Return ConsoleWrite("$oEdit ERR" & @CRLF)
    ConsoleWrite("$oEdit OK" & @CRLF)

    ; --- Value Pattern (action) Object ---

    ConsoleWrite("--- Value Pattern (action) Object ---" & @CRLF)

    Local $pValuePattern, $oValuePattern
    $oEdit.GetCurrentPattern($UIA_ValuePatternId, $pValuePattern)
    $oValuePattern = ObjCreateInterface($pValuePattern, $sIID_IUIAutomationValuePattern, $dtag_IUIAutomationValuePattern)
    If Not IsObj($oValuePattern) Then Return ConsoleWrite("$oValuePattern ERR" & @CRLF)
    ConsoleWrite("$oValuePattern OK" & @CRLF)

    $oValuePattern.SetValue("AutoIt rules")

EndFunc   ;==>Example

The console output shows that I can reach the green underlined window, document, but I can't get to the Edit control:

$oUIAutomation OK
$oDesktop OK
--- Find window ---
$pCondition OK
$oWindow OK
--- Find document ---
$pCondition OK
$oDocument OK
--- Find control ---
$pCondition OK
$oEdit ERR

I've tried to add a (possibly pointless) step to access the Document control, as in the code, but the result is the same.

Is there a limit to the maximum supported depth between a window (or here a document) and the targeted Edit? I notice there are a large number of Group controls in between.