Get e-mail address from selected e-mail message in Thunderbird

[maniootek]

Autoit Window Info gives no information about Thunderbird window exept title and class which is not enough for me. I would like to read some controls from it to get more info like e-mail address from selected message. I have found some script which extract all information from the handle of specified window to txt file, which is based on UIAutomation UDF from this thread:

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/

this is my part of code:

expandcollapse popup

#include "CUIAutomation2.au3"

Opt("MustDeclareVars", 1)
Opt("WinTitleMatchMode", 2)

Global $oUIAutomation
Global $file = "result.txt"

MainFunc()

Func MainFunc()

    ;Local $hWindow = 0x0006045C                                                             ; Infragistics
    ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Infragistics
    ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )                                 ; Windows Explorer, Windows 7
    ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )                                ; Windows Explorer, Windows XP
    ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )                          ; Windows Explorer right pane
    ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" )         ; AutoIt GUI window
    ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" )                                ; AutoIt script
    ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                                      ; Internet Explorer
    
    Local $hWindow = WinGetHandle("Thunderbird")
    If Not $hWindow Then Return

    $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return

    Local $pWindow
    ;$oUIAutomation.GetRootElement( $pWindow )             ; Desktop
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow) ; Window
    If Not $pWindow Then Return

    Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return

    ;ListDescendants( $oWindow, 0, 1 ) ; Desktop
    ListDescendants($oWindow, 0, 0) ; Window

EndFunc   ;==>MainFunc


Func ListDescendants($oParent, $iLevel, $iLevels = 0)

    If Not IsObj($oParent) Then Return
    If $iLevels And $iLevel = $iLevels Then Return

    Local $pRawWalker, $oRawWalker
    $oUIAutomation.RawViewWalker($pRawWalker)
    $oRawWalker = ObjCreateInterface($pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)

    Local $pUIElement, $oUIElement
    $oRawWalker.GetFirstChildElement($oParent, $pUIElement)
    $oUIElement = ObjCreateInterface($pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    Local $sIndent = ""
    For $i = 0 To $iLevel - 1
        $sIndent &= "    "
    Next

    While IsObj($oUIElement)

        FileWrite($file, $sIndent & "Title     = " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @CRLF & _
                $sIndent & "Class     = " & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & @CRLF & _
                $sIndent & "Ctrl type = " & _UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId) & @CRLF & _
                $sIndent & "Ctrl name = " & _UIA_getPropertyValue($oUIElement, $UIA_LocalizedControlTypePropertyId) & @CRLF & _
                $sIndent & "Value     = " & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleValuePropertyId) & @CRLF & _
                $sIndent & "Handle    = " & Hex(_UIA_getPropertyValue($oUIElement, $UIA_NativeWindowHandlePropertyId)) & @CRLF & @CRLF)

        ListDescendants($oUIElement, $iLevel + 1, $iLevels)

        $oRawWalker.GetNextSiblingElement($oUIElement, $pUIElement)
        $oUIElement = ObjCreateInterface($pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    WEnd

EndFunc   ;==>ListDescendants


Func _UIA_getPropertyValue($obj, $id)
    Local $tVal
    $obj.GetCurrentPropertyValue($id, $tVal)
    If Not IsArray($tVal) Then Return $tVal
    Local $tStr = $tVal[0]
    For $i = 1 To UBound($tVal) - 1
        $tStr &= "; " & $tVal[$i]
    Next
    Return $tStr
EndFunc   ;==>_UIA_getPropertyValue

in the result.txt file I have found a lot of information extracted from thunderbird window like this:

Quote

                Title     = Od: Malgorzata Nowak <malgorzatanowak@domain.com>
                Class     =
                Ctrl type = 50025
                Ctrl name =
                Value     =
                Handle    = 00000000

which states "e-mail address from field" from selected message

Now, how can I get this specified type of information directly? Is there some function to read specified ctrl type? I just want my script working faster without proces this huge part of returned information.

Edit:

I just realised that there is already function to read these values "_UIA_getPropertyValue", but it still too slow to get it. Full window read takes about 4.5sec, when i modify the script to look for ctrl type "50025" and title which contains "od: " then it takes 1.5sec which is still not enought for me.

Edited April 2, 2016 by maniootek

[mLipok]

Save email to EML file

And you can parse it or use COM object to access EML properties

here is some related stuff

 

but it need to do some work with this.

 

 

 

 

[maniootek]

this is more about read from the window rather than read from the file

[LarsJ]

The code in post 1 should only be used to print the entire tree structure. It should not be used to search for a specific control.

To search for a specific control for example the Inbox you should use the "Simple spy demo" from the UI Automation framework or Inspect.exe from Windows SDK to identify the control. When you have identified the control you can set up a property condition to search directly for this control. The Inbox is probably some kind of a list control. Then you can get information for all or selected items in the list.

[maniootek]

Thank you for your message.

I have tried Inspect.exe and simplespy.au3 and this is result

Inspect.exe:

 

simplespy.au3:

expandcollapse popup

Mouse position is retrieved 262-584
At least we have an element [Od: Dysk Google <drive-noreply@google.com>][]

Having the following values for all properties: 
Title is: <Od: Dysk Google <drive-noreply@google.com>>  Class   := <>   controltype:= <UIA_EditControlTypeId>   ,<50004>    , (0000C354)    207;566;250;20
*** Parent Information top down ***
6: Title is: <Pulpit>   Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1920;1200
"Title:=Pulpit;controltype:=UIA_PaneControlTypeId;class:=#32769"    
5: Title is: <Odebrane - email@gmail.com - Mozilla Thunderbird> Class   := <MozillaWindowClass> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    -8;-8;1936;1176
"Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass"    
4: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    0;54;1562;1084
"Title:=;controltype:=UIA_CustomControlTypeId;class:="  
3: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    0;54;1562;1084
"Title:=;controltype:=UIA_CustomControlTypeId;class:="  
2: Title is: <Od>   Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    207;565;250;23
"Title:=Od;controltype:=UIA_CustomControlTypeId;class:="    
1: Title is: <> Class   := <>   controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    207;565;250;22
"Title:=;controltype:=UIA_TextControlTypeId;class:="    
0: Title is: <Od: Dysk Google <drive-noreply@google.com>>   Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    207;566;250;20
"Title:=Od: Dysk Google <drive-noreply@google.com>;controltype:=UIA_CustomControlTypeId;class:="    


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP5=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)    
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Od;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) 
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_TextControlTypeId;class:=", $treescope_children) 
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Od: Dysk Google <drive-noreply@google.com>;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) 
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Od:DyskGoogle<drive-.mainwindow", "title:=Od: Dysk Google <drive-noreply@google.com>;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Od: Dysk Google <drive-noreply@google.com>;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_text:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_regexptitle:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_iaccessiblevalue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <207;566;250;20>
UIA_ProcessId:= <9188>
UIA_ControlType:= <50004>
UIA_LocalizedControlType:= <edytuj>
UIA_Name:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <True>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueValue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Od: Dysk Google <drive-noreply@google.com>>
UIA_LegacyIAccessibleValue:= <Dysk Google <drive-noreply@google.com> Dodaj do książki adresowej>
UIA_LegacyIAccessibleRole:= <42>
UIA_LegacyIAccessibleState:= <1074790464>
UIA_LegacyIAccessibleDefaultAction:= <click>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:9188,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

now, how can I access and get title from that control?

Edited April 2, 2016 by maniootek

[LarsJ]

If you look at the output from "Simple spy demo" the chain of parent controls contains only one UIA_TextControlTypeId. Try to set up a property condition (with CreatePropertyCondition method) to search for this text control.

[maniootek]

This code seems to work:

#include "UIAWrappers.au3"

Local $oP5=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:= - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Od;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_TextControlTypeId;class:=", $treescope_children)
_UIA_action($oP1, "click")

but how to get title of $oP1?

tried this but does not work

MsgBox(0,0,_UIA_getPropertyValue($oP1, $UIA_NamePropertyId))

Also is there other way to process this code faster? I have changed UIA.cfg file to do not mark on red but it still do it. Any chance to get this title value even when Thunderbird is not in focus (from the background) ?

Edited April 3, 2016 by maniootek

[LarsJ]

If you've looked at UIAWrappers.au3 you've seen that the functions in this UDF executes a lot of code. To execute a smaller amount of code and make it faster search for CreatePropertyCondition in some of my examples. The Thunderbird window does not need to have focus.

I'll give you a little time to do some reading and testing. I'll take a new look at this thread next sunday.

[junkew]

Speed is a relative problem

A. Faster cpu

B. Switch to compiled language

C. Study all examples.zip to not use uiawrappers but loose flexibility

D. Wait till I make a firebird example but that takes longer

In general first make an object highlight and after that make another action. Your msgbox example normally should work.

[junkew]

Not on my pc so hard to make example but try this

1. Get top level window as treescope child from desktop

2. Get element as treescope subtree from top level window based on title property

3. Highlight element

Speed can be faster by finding with 1 property using function _UIA_getFirstObjectOfElement

 

 

[maniootek]

I have this part of code so far:

#include "CUIAutomation2.au3"

Opt("WinTitleMatchMode", 2)

Local $hWindow = WinGetHandle(" - Mozilla Thunderbird")
$oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
Local $pWindow
$oUIAutomation.ElementFromHandle($hWindow, $pWindow) ; Window
Local $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

what metod for $oWindow object should I use to find element with controltype "UIA_TextControlTypeId" ?

 

Edited April 10, 2016 by maniootek

[MichaelHB]

Sorry if i misunderstood your intent here. I belive you need to get the text of a specific part of the Thunderbird, and that is retrived by the "title" right?

Try something like this:

Notice that you will need to get the $oUIElement first.

_UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)

Look if this retrieve the data you want.

[maniootek]

 

20 minutes ago, MichaelHB said:

Notice that you will need to get the $oUIElement first.

the question now is how to get this element first, i know that title contains text started with "Od: " and controltype is 50025 (which I think means "UIA_CustomControlTypeId") and that parent element controltype is "UIA_TextControlTypeId" (which I think means 50020)

I have noticed that this element (controltype 50020) only occur 2 times before my target. First 50020 element has title "Od" and the second one has empty title. The second one is parent of my target. My target is first child of this parent.

 

 

Edited April 10, 2016 by maniootek

[MichaelHB]

Use simplespy to get the element near/parent to the data you want to extract and then use something like this. This is the same function that is in the simplespy.

#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"


MsgBox(0, "", Retrieve())

Func Retrieve()
    Local $oParentHandle[1]

    Local $Thunderbird = _UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Odebrane - email@gmail.com - Mozilla Thunderbird;controltype:=UIA_WindowControlTypeId;class:=MozillaWindowClass", $treescope_children)    
    _UIA_Action($oP5,"setfocus")
    Local $oUIElement = _UIA_getObjectByFindAll($Thunderbird, "*** TITLE AND TYPE ***", $treescope_subtree) ; <= you need to fill this part

    $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
    Local $oTW = ObjCreateInterface($UIA_pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)
    If IsObj($oTW) = 0 Then
        MsgBox(0, "UI automation treewalker failed", "UI Automation failed failed")
    EndIf

    $oTW.GetParentElement($oUIElement, $oParentHandle[0])
    $oParentHandle[0] = ObjCreateInterface($oParentHandle[0], $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If IsObj($oParentHandle[0]) = 0 Then
        MsgBox(0, "No parent", "UI Automation failed could be you spy the desktop")
    Else
        Return _UIA_getPropertyValue($oParentHandle[0], $UIA_NamePropertyId)
    EndIf

EndFunc

Exit

 

Edited April 10, 2016 by MichaelHB
Changed the code.

[maniootek]

My goal is to save the time when working on UIA that's why I don't want to use UIAWrappers (as @junkew advice). I don't want to use

_UIA_getObjectByFindAll

or 

_UIA_Action

 

[LarsJ]

You can do some tests with this code

expandcollapse popup

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )
Opt( "WinTitleMatchMode", 2 )

Example()

Func Example()
  ; Mozilla Thunderbird window
  Local $hWindow = WinGetHandle( " - Mozilla Thunderbird" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  ; Note that $oWindow is an AutomationElement object
  ; Search for $dtagIUIAutomationElement in CUIAutomation2.au3 to see which methods can be used
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  #cs
  ; Condition to find text controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )
  #ce

  ; Condition to find custom controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition )
  ; You can use any of the ControlTypeIds in CUIAutomation2.au3 (search for ControlTypeId)
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find custom controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  ; You can use any of the TreeScope constants in CUIAutomation2.au3 (search for TreeScope)
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find controls ERR" & @CRLF )
  ConsoleWrite( "Find controls OK" & @CRLF )

  ; Print custom control info
  Local $pUIElement, $oUIElement
  For $i = 0 To $iElements - 1
    ConsoleWrite( @CRLF )
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    ConsoleWrite( "Index     = " & $i & @CRLF & _
                  "Name      = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Rectangle = " & GetCurrentPropertyValue( $oUIElement, $UIA_BoundingRectanglePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF )
                  ; You can add any of the PropertyIdIds in CUIAutomation2.au3 (search for PropertyId)
  Next

EndFunc

Func GetCurrentPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

 

[maniootek]

Thank you so much for your example, it is very clear. Anyway, I still have some questions:

1. Where can I find list of methods for $oWindow (AutomationElement object), so far i know .FindAll and .FindFirst

2. What would be fastest way to find element which $UIA_NamePropertyId contains "Reply to" value and which is $UIA_CustomControlTypeId (50025)

3. My target element is a child of second occurrence of $UIA_TextControlTypeId (50020) - would be that helpful to speed up searching my target element?

This code is processed in Time: 0.7706 which is much faster than my previous solution but I think it could be still improved:

expandcollapse popup

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )
Opt( "WinTitleMatchMode", 2 )

Example()

Func Example()
  ; Mozilla Thunderbird window
  Local $hWindow = WinGetHandle( " - Mozilla Thunderbird" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  ; Note that $oWindow is an AutomationElement object
  ; Search for $dtagIUIAutomationElement in CUIAutomation2.au3 to see which methods can be used
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find custom controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition );$UIA_CustomControlTypeId
  ; You can use any of the ControlTypeIds in CUIAutomation2.au3 (search for ControlTypeId)
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find custom controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  ; You can use any of the TreeScope constants in CUIAutomation2.au3 (search for TreeScope)
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find controls ERR" & @CRLF )
  ConsoleWrite( "Find controls OK" & @CRLF )

  ; Print custom control info
  Local $pUIElement, $oUIElement, $name
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $name = GetCurrentPropertyValue($oUIElement,$UIA_NamePropertyId)
    if StringInStr($name, "Reply to:") Then
        ConsoleWrite($name & @CRLF)
        Exit
    EndIf
  Next

EndFunc

Func GetCurrentPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

 

[LarsJ]

From CUIAutomation2.au3. This is all the methods and properties:

expandcollapse popup

Global Const $sIID_IUIAutomationElement="{D22108AA-8AC5-49A5-837B-37BBB3D7591E}"
Global $dtagIUIAutomationElement = "SetFocus hresult();" & _
"GetRuntimeId hresult(ptr*);" & _
"FindFirst hresult(long;ptr;ptr*);" & _
"FindAll hresult(long;ptr;ptr*);" & _
"FindFirstBuildCache hresult(long;ptr;ptr;ptr*);" & _
"FindAllBuildCache hresult(long;ptr;ptr;ptr*);" & _
"BuildUpdatedCache hresult(ptr;ptr*);" & _
"GetCurrentPropertyValue hresult(int;variant*);" & _
"GetCurrentPropertyValueEx hresult(int;long;variant*);" & _
"GetCachedPropertyValue hresult(int;variant*);" & _
"GetCachedPropertyValueEx hresult(int;long;variant*);" & _
"GetCurrentPatternAs hresult(int;none;none*);" & _
"GetCachedPatternAs hresult(int;none;none*);" & _
"GetCurrentPattern hresult(int;ptr*);" & _
"GetCachedPattern hresult(int;ptr*);" & _
"GetCachedParent hresult(ptr*);" & _
"GetCachedChildren hresult(ptr*);" & _
"CurrentProcessId hresult(int*);" & _
"CurrentControlType hresult(int*);" & _
"CurrentLocalizedControlType hresult(bstr*);" & _
"CurrentName hresult(bstr*);" & _
"CurrentAcceleratorKey hresult(bstr*);" & _
"CurrentAccessKey hresult(bstr*);" & _
"CurrentHasKeyboardFocus hresult(long*);" & _
"CurrentIsKeyboardFocusable hresult(long*);" & _
"CurrentIsEnabled hresult(long*);" & _
"CurrentAutomationId hresult(bstr*);" & _
"CurrentClassName hresult(bstr*);" & _
"CurrentHelpText hresult(bstr*);" & _
"CurrentCulture hresult(int*);" & _
"CurrentIsControlElement hresult(long*);" & _
"CurrentIsContentElement hresult(long*);" & _
"CurrentIsPassword hresult(long*);" & _
"CurrentNativeWindowHandle hresult(hwnd*);" & _
"CurrentItemType hresult(bstr*);" & _
"CurrentIsOffscreen hresult(long*);" & _
"CurrentOrientation hresult(long*);" & _
"CurrentFrameworkId hresult(bstr*);" & _
"CurrentIsRequiredForForm hresult(long*);" & _
"CurrentItemStatus hresult(bstr*);" & _
"CurrentBoundingRectangle hresult(struct*);" & _
"CurrentLabeledBy hresult(ptr*);" & _
"CurrentAriaRole hresult(bstr*);" & _
"CurrentAriaProperties hresult(bstr*);" & _
"CurrentIsDataValidForForm hresult(long*);" & _
"CurrentControllerFor hresult(ptr*);" & _
"CurrentDescribedBy hresult(ptr*);" & _
"CurrentFlowsTo hresult(ptr*);" & _
"CurrentProviderDescription hresult(bstr*);" & _
"CachedProcessId hresult(int*);" & _
"CachedControlType hresult(int*);" & _
"CachedLocalizedControlType hresult(bstr*);" & _
"CachedName hresult(bstr*);" & _
"CachedAcceleratorKey hresult(bstr*);" & _
"CachedAccessKey hresult(bstr*);" & _
"CachedHasKeyboardFocus hresult(long*);" & _
"CachedIsKeyboardFocusable hresult(long*);" & _
"CachedIsEnabled hresult(long*);" & _
"CachedAutomationId hresult(bstr*);" & _
"CachedClassName hresult(bstr*);" & _
"CachedHelpText hresult(bstr*);" & _
"CachedCulture hresult(int*);" & _
"CachedIsControlElement hresult(long*);" & _
"CachedIsContentElement hresult(long*);" & _
"CachedIsPassword hresult(long*);" & _
"CachedNativeWindowHandle hresult(hwnd*);" & _
"CachedItemType hresult(bstr*);" & _
"CachedIsOffscreen hresult(long*);" & _
"CachedOrientation hresult(long*);" & _
"CachedFrameworkId hresult(bstr*);" & _
"CachedIsRequiredForForm hresult(long*);" & _
"CachedItemStatus hresult(bstr*);" & _
"CachedBoundingRectangle hresult(struct*);" & _
"CachedLabeledBy hresult(ptr*);" & _
"CachedAriaRole hresult(bstr*);" & _
"CachedAriaProperties hresult(bstr*);" & _
"CachedIsDataValidForForm hresult(long*);" & _
"CachedControllerFor hresult(ptr*);" & _
"CachedDescribedBy hresult(ptr*);" & _
"CachedFlowsTo hresult(ptr*);" & _
"CachedProviderDescription hresult(bstr*);" & _
"GetClickablePoint hresult(struct*;long*);"

And the MicroSoft documentation is here.

With $UIA_NamePropertyId you can only search for the entire name.

If you can find an object or element with a single FindAll or FindFirst command this is normally the fastest.

Did you notice that I've added the index in the console output in the code in post 16. This index is usually the same all the time. When you know the index you don't need the For loop. This can save a lot of time.

[maniootek]

This is good advice about index of array. Anyway I did more research about speed up my script and look my results

Ctrl type 50025 occurs 268 times when 50020 (which contains my target title)  only 20 times. I tried to change

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CustomControlTypeId, $pCondition ) ;50025 (find controls ok)

to:

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition ) ;50020 (find controls err)

or 

$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition ) ;50004 (find controls err)

 

but console output says:

Quote

Find controls ERR

What could be wrong? I have noticed that you already used this part of new code in your example (between comments tags #cs and #ce)

Edited April 12, 2016 by maniootek

[LarsJ]

I don't think there is anything wrong. But there is no direct edit or text control descendants of the window object. The edit and text controls are probably child elements of the custom controls.

How many custom controls have been found with the code in post 17? What's the value of $iElements? The figure should be quite high before it pays to make more partial searches. More partial searches requires more AutoIt code, while the FindAll command is running in the UI Automation dll with the speed of compiled C++ code.

Edited April 12, 2016 by LarsJ