Using UI Automation Code in AutoIt

[junkew]

There is no limit. But dealing with the hierarchy can be troublesome.

I frequently highlight controls during development or do a findall in a subtree which can be slower but is programming wise easier certainly when you start with uia.

Dealing with html can potentially be easier with webdriver but both have a challenging learning curve.

[jchd]

Thanks, but do you have any idea why the Edit isn't found? The $UIA_AutomationIdPropertyId of that edit control is correct: addressFormDestPart.lastName

[Nine]

Try removing the document part, and search from the window instead...

[jchd]

That's what I tried first, without success either.

I tried to get to the first level Group under Document (which is found OK every time), but the group isn't found!

Subsequent tries show that no Group (tried all having non-empty $UIA_AutomationIdPropertyId) below Document can be found. Searching from Windows also fails, but takes more time that from Document, unsurprisingly.

Is the current UDF able to detect Group controls at all?

Here's the ancestors tree from the first Edit I want to set data into:

[Danyfirex]

For sure it works with Group using the condition: $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_GroupControlTypeId, $pCondition)

are you using correct Arch x64/x86 to be sure is not related to it.

 

Edit:

Can You try to find all edits with condition $UIA_ControlTypePropertyId, $UIA_EditControlTypeId?

Saludos

Edited April 12 by Danyfirex

[jchd]

Old stock desktop Win10 version 22H2 x64, AutoIt v3.3.16.1 code running as x32, 16Gb RAM.

 

[Danyfirex]

@jchd I meant make sure Your script and the Firefox instance are both same arch. (just to skip it could be the issue)

also Can You try to find all edits with condition $UIA_ControlTypePropertyId, $UIA_EditControlTypeId?

 

 

Saludos

[jchd]

Also tried x64 code as FF is x64, but still no joy.

.../...

    ; --- Find all edit controls ---

    ConsoleWrite("--- Find all edit controls ---" & @CRLF)

    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("$pCondition ERR" & @CRLF)
    ConsoleWrite("$pCondition OK" & @CRLF)

    Local $pEditAll, $oEditAll
    $oWindow.FindAll($TreeScope_Descendants, $pCondition, $pEditAll)
    $oEditAll = ObjCreateInterface($pEditAll, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
    If Not IsObj($oEditAll) Then Return ConsoleWrite("$oEditAll ERR" & @CRLF)
    ConsoleWrite("$oEditAll OK" & @CRLF)

Still error, whether parent is Window or Document:

$oUIAutomation OK
$oDesktop OK
--- Find window ---
$pCondition OK
$oWindow OK
--- Find document ---
$pCondition OK
$oDocument OK
--- Find all edit controls ---
$pCondition OK
$oEditAll ERR

Also tried $TreeScope_Children instead of $TreeScope_Descendants (I don't know if that should make a difference).

[jchd]

There are a number of Text controls, but FindAll fails on them.

[jchd]

I first thought I could go with a series of Send, but sending Tab to switch to the next edit applies to the menu and things break down.

This page is nothing fancy: it's French post form to create packets shipment forms. I can't believe they use anything to protect against automation, since this would be contrary to their business!

[SOLVE-SMART]

Would you try or create a comparison by using the WebDriver project "au3WebDriver"?
Only to see how far you would came with this approach 🤔 .

Don't get me wrong @jchd, I like UIA alot (when proper wrapper functions make your live with it easier 😅), but like @junkew already said, sometimes the WebDriver approach is more feasable then UIA. Of course there are plenty of cases where it's the other way around, but I am really curious about the "what if".

Best regards
Sven

[Nine]

@jchd I think it should be :

$oWindow.FindAll($TreeScope_Descendants, $oCondition, $pElement)
$oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

 

[jchd]

Right, but still no!

; --- Find all edit controls ---

    ConsoleWrite("--- Find all edit controls ---" & @CRLF)

    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("$pCondition ERR" & @CRLF)
    ConsoleWrite("$pCondition OK" & @CRLF)

    Local $pEditAll, $oEditAll
    $oWindow.FindAll($TreeScope_Descendants, $pCondition, $pEditAll)
    $oEditAll = ObjCreateInterface($pEditAll, $sIID_IUIAutomationElement, $dtag_IUIAutomationElementArray)
    If Not IsObj($oEditAll) Then Return ConsoleWrite("$oEditAll ERR" & @CRLF)
    ConsoleWrite("$oEditAll OK" & @CRLF)

$oEditAll ERR

[Nine]

$sIID_IUIAutomationElementArray

[jchd]

Oops, my bad, this works for finding all text or all edit controls.

Now I guess all I need is find how to iterate within the edit array to find the relevant one(s). Digging in the UI threads...

[junkew]

You only need to dig into example 1 as part of the initial uia source that was shared. Understand the findall as that's easier then treewalking.

[jchd]

Finally I found what was the culprit: a very old FF window which got shifted at the extreme corner of a desktop, making it essentially invisible. The only tab it carried was the one I was after in another tab of another FF window, but was set to a different part of the website.

I confess having permanently 15-30 FF windows and many tabs each, all always open. Hard for me to do with much less.

So everything works with the initial simple code and I flatly apologize for the noise and for wasting your time.