IUIAutomation MS framework automate chrome, FF, IE, ....

[TSR80]

Hi Junkew,

The problem is that I cannot find the info in the automation objects through inspect.exe or Simple UI Spy.  I can only see the values through visual inspection of the grid control in the user interface.

[junkew]

what did you mean with

 

 of the grid control are visible in the UIAutomation tree

 

To me that looked like you are looking at a UIAutomation tree with some kind of a tool?

But if you did find any usefull information with tools I doubt if its possible

Suggestions I have for other tools to try are

1. NVDA (screen reader open source)
2. http://www.freedomscientific.com/Products/Blindness/JAWS
3. http://www.ranorex.com/gui-testing-guide/spy.html
   If these all do not give information it will probably be hard to get what you want but commercial free trials could help
4. http://smartbear.com/product/testcomplete/free-trial/
   and this fifth one if you really are going to dig deep into an application (not recommended)
5. http://www.nektra.com/products/deviare-api-hook-windows/

[TSR80]

Oh, what I meant was that I can visually see the values that I want to read into my script in the grid control of the application.

Using inspect.exe, I can also navigate to the grid control (ControlType = UIADataGridControlTypeId) and its children (including UIA data item control type and some custom control types).  However, none of the properties of any of the objects I can navigate to in the object tree contain the values I can visually see in the grid control.  Thus, I can't figure out a way to load the values into my script.

I'll try some of the other tools to see if there might be additional data that is not being shown in inspect.exe.  Thanks!

[TSR80]

I have yet another question that I'm hoping somebody (Junkew?) can answer for me.  I hope I'm not being too much of a bother!

Using inspect.exe, I can view the RuntimeId property of each row in a grid control.  According to the MSDN documentation, the RuntimeId property is an array of three int32 values.   So, if I get the value of the RuntimeId property of a particular row and write it to the console, it is displayed like this: "7;4988;63749147."  If I view the same RuntimeId property of the same row in inspect.exe, it is displayed like this: [7.137C.3CCBC1B].

In other words, inspect.exe is just showing me the hexadecimal values of each of the three integers, whereas the console is showing me the decimal values of the three integers.  So far so good.

Now when a row is removed from the grid, my structurechanged eventhandler is called.  Here is the prototype of the function, as taken from the examples:

Func oIUIAutomationStructureChangedEventHandler_HandleStructureChangedEvent( $pSelf, $pSender, $iChangeType, $pRuntimeId ) ; Ret: long  Par: ptr;long;ptr

According to the MSDN documentation, when the iChangeType indicates a child has been removed, the pRuntimeId parameter is set to the RuntimeId of the child that was removed.

Thus, to determine which row has been removed from my grid control, I would like to match the pRuntimeId parameter that is passed into the event handler (and which also identifies the RuntimeId of the object that has been removed) to one of RuntimeIds of the rows I have previously identified as having been added to the grid control.

However, the pRuntimeId is a pointer to the RuntimeId.  So, for example, printing the value of the pRuntimeId parameter to the console gives me a hexadecimal value that I think is the address of the pointer, not the values of the three integers that the pointer is pointing to.  So I get something like this when I print the value of the pRuntimeId parameter of the event handler to the console: 0x042BC8A8.

How do I get the three integer values that the pointer is pointing to so that I can do the comparison in my event handler?

Once I'm able to get the three integer values from the pRuntimeId pointer passed into the event handler, I think I'll be able to use the IUIAutomation::CompareRuntimeIds method (in conjunction with the safearray UDF) to do the comparison.  But I need to get the values from the pointer first.  This would be relatively straightforward in C, but I'm not understanding how to do it in AutoIt.

I believe an answer to this question might also solve the problem I noted with getting the value of a VARIANT parameter that is passed into a different event handler (see post #337).

Edited January 22, 2015 by TSR80

[TSR80]

Aha.  It looks like LarsJ may have answered my question back in post #172.

I should be able to access the elements of the safe array using the safearray UDF.

I'll let you know if this works.

Edited January 23, 2015 by TSR80

[TSR80]

BINGO! 

You can pass the $pRuntimeID parameter directly to the SafeArrayGetElement function defined in SafeArray.au3.

It looks like there is an incorrectly-specified type in that function though.  According to the MSDN documentation, the HRESULT returned by the SafeArrayGetElement function in the ole32.dll is a long, not an int.

So I made the following change to the DllCall line in the SafeArrayGetElement function in SafeArray.au3:

I changed

Local $aRet = DllCall( "oleaut32.dll", "int", "SafeArrayGetElement", "ptr", $pSafeArray, "long*", $iIndex, $sType, 0)

to

Local $aRet = DllCall( "oleaut32.dll", "long", "SafeArrayGetElement", "ptr", $pSafeArray, "long*", $iIndex, $sType, 0)

Here's some sample code for my event handler (not compilable):

expandcollapse popup

Func oIUIAutomationStructureChangedEventHandler_HandleStructureChangedEvent( $pSelf, $pSender, $iChangeType, $pRuntimeId ) ; Ret: long  Par: ptr;long;ptr
#
  Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSender.AddRef()

  ; Ignore events that aren't related to the processes associated with your executable.

   If StringInStr(__WinAPI_GetProcessFileName(_UIA_getPropertyValue( $oSender, $UIA_ProcessIdPropertyId )), '<filename.exe>') Then

      ; ConsoleWrite( @CRLF & "oIUIAutomationStructureChangedEventHandler_HandleStructureChangedEvent: " & $pSender & " " & $iChangeType & @CRLF )
      Switch $iChangeType
         Case $StructureChangeType_ChildAdded
            ; ConsoleWrite( " StructureChangeType=ChildAdded: " & @CRLF )

                 ; $StructureChangeType_ChildRemoved is the only $iChangeType for which the $pRuntimeId parameter is not void.
         Case $StructureChangeType_ChildRemoved
            ; ConsoleWrite( " StructureChangeType=ChildRemoved: " & @CRLF )
            Local $sAutomationIdPropertyValue = _UIA_getPropertyValue( $oSender, $UIA_AutomationIdPropertyId)
            ; Make sure this is the parent you are interested in (e.g., check the AutomationId property of $oSender.
                        Local $sAutomationPropertyValue = _UIA_getPropertyValue( $oSender, $UIA_AutomationIdPropertyId )f
                        If ($sAutomationIdPropertyValue = "<propertyvalue>") Then
               Local $pValue0, $pValue1, $pValue2
 
                           ; Get the three values referenced from the pRuntimeId SAFEARRAY.
               SafeArrayGetElement( $pRuntimeId, 0, $pValue0 )
               ; ConsoleWrite($pValue0 & @CRLF)
               SafeArrayGetElement( $pRuntimeId, 1, $pValue1 )
               ; ConsoleWrite($pValue1 & @CRLF)
               SafeArrayGetElement( $pRuntimeId, 2, $pValue2 )
               ; ConsoleWrite($pValue0 & @CRLF)

                           ; Make the format of the values match the format of the string returned from _UIA_getPropertyValue( $oSender, $UIA_RunTimeIdPropertyId (called when the row is added)
               Local $sConstructedRunTimeId = $pValue0 & ";" & $pValue1 & ";" & $pValue2
                           ; Here, do matching of $sContructionRunTimeId to previous RuntimeIDs collected

            EndIf
         Case $StructureChangeType_ChildrenInvalidated
            ; ConsoleWrite( " StructureChangeType=ChildrenInvalidated: " & @CRLF )
         Case $StructureChangeType_ChildrenBulkAdded
            ; ConsoleWrite( " StructureChangeType=ChildrednBulkAdded: " & @CRLF )
         Case $StructureChangeType_ChildrenBulkRemoved
            ; ConsoleWrite( " StructureChangeType=ChildrenBulkRemoved: " & @CRLF )
         Case $StructureChangeType_ChildrenReordered
            ; ConsoleWrite( " StructureChangeType=ChildrenReordered: " & @CRLF )
      EndSwitch

   EndIf
   Return $S_OK
EndFunc

Edited January 23, 2015 by TSR80

[LarsJ]

TSR80, Very nice.

[TSR80]

Thanks LarsJ!  You saved me a lot of time with the safearray UDF.

I'm still not sure how to correctly read the variant $newValue parameter that is passed into my oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent event handler.  (See post #337.)

Any idea how to read that value?

Edited January 23, 2015 by TSR80

[LarsJ]

I have been playing with the $newValue parameter without success.

In AutoIt a variant can be described with this structure:

Global Const $tagVARIANT = "word vt;word r1;word r2;word r3;ptr data; ptr;"

vt is the variant type

data is a pointer to the data

If vt = VT_I4 (= 3, signed 4-byte integer) you'll get the data like this:

$data = DllStructGetData( $tVARIANT, "data" )
$tINT = DllStructCreate( "int", $data )
$int  = DllStructGetData( $tINT, 1 )

When I run the code in post 336 (#1220187) I'm getting a lot of $newValue = 0x00000008 values. This is a suspiciously low memory address, and when I try to create the structure:

$tVARIANT = DllStructCreate( $tagVARIANT, $newValue )

AutoIt freezes, and I have to kill the process.

If I switch between different windows, I'll get higher values of $newValue. Then AutoIt doesn't freeze, but the vt values are either VT_NULL or invalid values.

Because $iPropertyId = 30005 in all cases and 30005 means UIA_NamePropertyId which is represented as a BSTR, I would expect a value of vt = VT_BSTR (= 8).

I've searched the internet without finding any valuable information. I don't think you can get any information directly from the $newValue parameter.

Your code in post 336 is working really good. When I was typing this text in Notepad the script was still running. I constantly got "Ln 19, Col 1", "Ln 19, Col 22", "Ln 20, Col 1" events.

[Proph]

OK... forgive me.  I know I am about to ask a dumb question...

I want to attempt to automate a program but I am not sure how. 

I can use your simplespy.au3 script and it gives me a ton of useful information.  What I am wondering (and hoping) is that it is giving me info I could use with the ControlClick function?

For instance... I'll attach the output given to me from simplespy.au3.  It recognizes the button I want to use controlclick on.  What info would I grab out of this to put into ControlClick?  Or do I need to use a totally different function besides ControlClick to do what I want to do?

 

Edit:  OK  I see that in the output from simplespy.au3 it list code to click the button in the ;~ *** Standard code *** section.  Which works for this button.  But now the second button I wish to click doesn't seem to be seen by simplespy.  Is there something I am doing wrong?  Or should I be doing something else to get the info for this second button?  I will attach a second simplespy_output2.txt file with what simple spy gives me.  But when I run the code it only clicks in the middle of the window instead of the button I wish to click.

SimpleSpy_output.txt

SimpleSpy_output2.txt

Edited January 28, 2015 by Proph

[junkew]

please ask question in general help and support. Most likely not possible with controlclick as its a qwidget

with iuiAutomation QWidgets are supported but apparently in your second output i already can see your control is not having a name as an propery to identify

you only can fallback to descriptions like: class:=qwidget;instance:=3 or use title:=<a textname that is there>;indexafter:=3 to refer to 3rd control after the control with the name <a textname that is there>

[Proph]

please ask question in general help and support. Most likely not possible with controlclick as its a qwidget

with iuiAutomation QWidgets are supported but apparently in your second output i already can see your control is not having a name as an propery to identify

you only can fallback to descriptions like: class:=qwidget;instance:=3 or use title:=<a textname that is there>;indexafter:=3 to refer to 3rd control after the control with the name <a textname that is there>

OK... I will post it in the support forum. Thanks!

[TSR80]

Thanks for trying to figure that one out LarsJ.  I guess it's not essential to be able read that value right now.  I think it could help with performance though.  I wonder is there is something wrong with the way the function handler is defined or being added?

I'm glad my code for the propertychanged event handler is working for you as well!

Edited January 29, 2015 by TSR80

[TSR80]

Junkew,

I think some of your functions should be updated to support caching (see the MSDN documentation here) .  For example, in my code, I would like to respond to a window created event by finding an object in the window and also caching the values of AutomationId properties of children of the object.  Then, when determining the AutomationId for these child objects later, I won't have to use a further cross-process communication.  Does that make sense?

I think some of your functions can be expanded pretty easily to include a Boolean parameter for specifying whether you wish to make a cached call or not.  But it looks like some supporting functions would be needed to activate the cache request.

What do you think?

Edited January 29, 2015 by TSR80

[junkew]

Whats your specific need for caching in your example

its on the long list as so far for speed I did not have the need to do caching.

. Getting the childs of the object in your example could also be done with a treewalker or findall (more or less caches also).

High level my thoughts are for the library to have 1 global setting caching=on and caching=off as i do not see a need to have a boolean per function to use caching yes/no

[TouchOdeath]

junkew, you sir are a genius, thank you thank you thank you for posting this!!  

EDIT:

Using #AutoIt3Wrapper_Run_Au3Stripper=y  gives me a ton of errors.

Edited January 29, 2015 by TouchOdeath

[LarsJ]

TSR80, For further tests of the $newValue parameter, I think it's necessary to use some C++ code for the test. In that way you can get rid of all AutoIt specific issues. If one can obtain the C++ code to work, it should be possible to translate into AutoIt.

You can find an example that shows how to use caching in post 92.

[junkew]

VT_BSTR = 0x0008 see https://msdn.microsoft.com/en-us/library/cc237865.aspx

Local $var = DllStructCreate($tagVARIANT, $newValue)

Local $VT = DllStructGetData($var, "vt")

 

then searching for VT_BSTR in the forums and reading AutoITObject.au3 you can get your string

 

Func _AutoItObject_VariantRead($pVariant)
    ; Author: monoceres, Prog@ndy
    Local $var = DllStructCreate($__Au3Obj_tagVARIANT, $pVariant), $data
    ; Translate the vt id to a autoit dllcall type
    Local $VT = DllStructGetData($var, "vt"), $type
    Switch $VT
        Case $__Au3Obj_VT_I1, $__Au3Obj_VT_UI1
            $type = "byte"
        Case $__Au3Obj_VT_I2
            $type = "short"
        Case $__Au3Obj_VT_I4
            $type = "int"
        Case $__Au3Obj_VT_I8
            $type = "int64"
        Case $__Au3Obj_VT_R4
            $type = "float"
        Case $__Au3Obj_VT_R8
            $type = "double"
        Case $__Au3Obj_VT_UI2
            $type = 'word'
        Case $__Au3Obj_VT_UI4
            $type = 'uint'
        Case $__Au3Obj_VT_UI8
            $type = 'uint64'
        Case $__Au3Obj_VT_BSTR
            Return __Au3Obj_SysReadString(DllStructGetData($var, "data"))

so probably if you include autoitobject.au3 and you do

consolewrite(_AutoItObject_VariantRead($newValue)) 

you will see something

Edited January 30, 2015 by junkew

[LarsJ]

That's exactly what I have done. But I got no valuable information.

Here is the code:

expandcollapse popup

#include "CUIAutomation2.au3"
#include "SafeArray.au3"
 
Opt( "MustDeclareVars", 1 )
 
Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"
Global Const $tagVARIANT = "word vt;word r1;word r2;word r3;ptr data; ptr;"
 
Global $tIUIAutomationPropertyChangedEventHandler, $oIUIAutomationPropertyChangedEventHandler
 
Global $oUIAutomation
 
MainFunc()
 
 
 
Func MainFunc()
 
  $oIUIAutomationPropertyChangedEventHandler = ObjectFromTag( "oIUIAutomationPropertyChangedEventHandler_", $dtagIUIAutomationPropertyChangedEventHandler, $tIUIAutomationPropertyChangedEventHandler, True )
  If Not IsObj( $oIUIAutomationPropertyChangedEventHandler ) Then Return
 
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return
 
  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return
 
#cs
  ; Use this code to call the AddPropertyChangedEventHandlerNativeArray method, which takes a normal array of property identifiers instead of a SAFEARRAY.
  ; Because of threading issues, calling this method instead of AddPropertyChangedEventHandler may lead to unexpected/incomplete results.
  Local $tPropertyArray = DllStructCreate( "int[2]" )
  DllStructSetData( $tPropertyArray, 1, $UIA_NamePropertyId, 1 )
  DllStructSetData( $tPropertyArray, 1, $UIA_ToggleToggleStatePropertyId, 2 )
  $oUIAutomation.AddPropertyChangedEventHandlerNativeArray( $pUIElement, $TreeScope_Descendants, 0, $oIUIAutomationPropertyChangedEventHandler, $tPropertyArray, 2 )
#ce
 
  ; Use this code to call the AddPropertyChangedEventHandlerNativeArray method, which takes a pointer to a SAFEARRAY of property identifiers.
  ; Because of threading issues, calling this method instead of AddPropertyChangedEventHandler appears to be preferable for automation of some applications.
 
  ; Create a SAFEARRAY vector having a number of rows equal to the number of property identifiers you will be monitoring for changes
  Local $ptrSafeArray = SafeArrayCreateVector( "int", 2 )
 
  ; Write SAFEARRAY structure
  Local Const $tagSAFEARRAY = "ushort cDims; ushort fFeatures; ulong cbElements; ulong cLocks; ptr pvData; ulong cElements; long lLbound"
  Local $tSAFEARRAY = DllStructCreate( $tagSAFEARRAY, $ptrSafeArray )
 
#cs
  ConsoleWrite( @CRLF & "SafeArray structure" & @CRLF )
  ConsoleWrite( "$tSAFEARRAY size       = " & DllStructGetSize( $tSAFEARRAY ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY cDims      = " & DllStructGetData( $tSAFEARRAY, "cDims" ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY fFeatures  = " & "0x" & Hex( DllStructGetData( $tSAFEARRAY, "fFeatures" ) ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY cbElements = " & DllStructGetData( $tSAFEARRAY, "cbElements" ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY cLocks     = " & DllStructGetData( $tSAFEARRAY, "cLocks" ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY pvData     = " & DllStructGetData( $tSAFEARRAY, "pvData" ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY cElements  = " & DllStructGetData( $tSAFEARRAY, "cElements" ) & @CRLF )
  ConsoleWrite( "$tSAFEARRAY lLbound    = " & DllStructGetData( $tSAFEARRAY, "lLbound" ) & @CRLF )
#ce
 
 
  ; Put the property identifiers corresponding to the properties you will be monitoring as elements into the SafeArray
  SafeArrayPutElement( $ptrSafeArray, 0, $UIA_NamePropertyId )
  SafeArrayPutElement( $ptrSafeArray, 1, $UIA_AutomationIdPropertyId )
 
  #cs
  Local $pValue
  ConsoleWrite( @CRLF & "Get the two conditions" & @CRLF )
  SafeArrayGetElement( $ptrSafeArray, 0, $pValue )
  ConsoleWrite( "int 1 = " &  $pValue & @CRLF )
  SafeArrayGetElement( $ptrSafeArray, 1, $pValue )
  ConsoleWrite( "int 2 = " &  $pValue & @CRLF )
  #ce
 
  ; Add the PropertyChangedEventHandler
  $oUIAutomation.AddPropertyChangedEventHandler( $pUIElement, $TreeScope_Descendants, 0, $oIUIAutomationPropertyChangedEventHandler, Ptr($ptrSafeArray))
 
  HotKeySet( "{ESC}", "Quit" )
 
  While Sleep(100)
  WEnd

  SafeArrayDestroy( $ptrSafeArray )
 
EndFunc
 
Func Quit()
  $oIUIAutomationPropertyChangedEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationPropertyChangedEventHandler )
  Exit
EndFunc
 
 
 
Func _UIA_getPropertyValue( $obj, $id )
  Local $vVal
  $obj.GetCurrentPropertyValue( $id, $vVal )
  Return $vVal
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent( $pSelf, $pSender, $iPropertyId, $newValue ) ; Ret: long  Par: ptr;int;variant
  ConsoleWrite( @CRLF & "oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent: $iPropertyId = " & $iPropertyId & ", $newValue = " & $newValue & @CRLF )

  If $newValue <> 0x00000008 Then
    Local $tVARIANT = DllStructCreate( $tagVARIANT, $newValue )
    Local $vt = DllStructGetData( $tVARIANT, "vt" )
    ConsoleWrite( "$vt = " & $vt & @CRLF )
  EndIf

    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
   
    ConsoleWrite( "AutomationId    " & _UIA_getPropertyValue( $oSender, $UIA_AutomationIdPropertyId ) & @CRLF )
    ConsoleWrite( "Handle    " & _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) & @CRLF )
    ConsoleWrite( "Name      " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF )
    ConsoleWrite( "Class     " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl type " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl name " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Value     " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF )
 
  Return $S_OK
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationPropertyChangedEventHandler Then
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUIAutomationPropertyChangedEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc
 
Func oIUIAutomationPropertyChangedEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc
 
Func oIUIAutomationPropertyChangedEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_Release" & @CRLF )
  Return 1
EndFunc
 
 
 
Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc
 
 
 
Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
            ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc
 
Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

Edited January 31, 2015 by LarsJ

[junkew]

• Is this a solution

  ConsoleWrite( "NewValue     " & _UIA_getPropertyValue( $oSender, $iPropertyId ) & @CRLF )

• maybe this is related:  

  http://community.nvda-project.org/ticket/3867

• this suggests to use the cached value

  https://social.msdn.microsoft.com/forums/windowsdesktop/en-us/ca163d9b-0a7d-452b-8954-b102d8d08447/registering-event-handler-for-changes-in-control-pattern-properties

• testing but the clock changed and gave a value

oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent: $iPropertyId = 30005, $newValue = 0x00010008
$vt = 47663
AutomationId    303
Handle    65710
Name      Klok
Class     TrayClockWClass
Ctrl type 50033
Ctrl name deelvenster
Value     17:29, ?zaterdag, ?31-?1-?2015

• A BSTR is actually a WCHAR* with a length prefix. The BSTR value points to the beginning of the string, not to the length prefix (which is stored in the bytes just “before” the location pointed to by the BSTR).

 

• accevent viewer tool (accessibility event viewer) gives this as output on notepad when entering text: "This is just a test!!!"

UIA:PropertyEvent   [ValueValue = "<unknown type>"] Sender: BoundingRectangle:{l:275 t:586 r:1402 b:1156}, ControlType:UIA_EditControlTypeId (0xC354), Name:"", AutomationId:"15", ClassName:"Edit", HelpText:"", ItemType:[Not supported], ValueValue:"this is just a test!!!", LegacyIAccessibleChildId:[not supported], LegacyIAccessibleName:[not supported], LegacyIAccessibleValue:[not supported], LegacyIAccessibleDescription:[not supported], LegacyIAccessibleRole:[not supported], LegacyIAccessibleState:[not supported], LegacyIAccessibleHelp:[not supported]
UIA:PropertyEvent   [Name = "   Ln 1, Col 23  "] Sender: BoundingRectangle:{l:1122 t:1158 r:1386 b:1179}, ControlType:UIA_TextControlTypeId (0xC364), Name:"   Ln 1, Col 23  ", AutomationId:[Not supported], ClassName:[Not supported], HelpText:"", ItemType:[Not supported], ValueValue:[Not supported], LegacyIAccessibleChildId:[not supported], LegacyIAccessibleName:[not supported], LegacyIAccessibleValue:[not supported], LegacyIAccessibleDescription:[not supported], LegacyIAccessibleRole:[not supported], LegacyIAccessibleState:[not supported], LegacyIAccessibleHelp:[not supported]
UIA:PropertyEvent   [Name = "   Ln 1, Col 23  "] Sender: BoundingRectangle:{l:1122 t:1158 r:1386 b:1179}, ControlType:UIA_TextControlTypeId (0xC364), Name:"   Ln 1, Col 23  ", AutomationId:[Not supported], ClassName:[Not supported], HelpText:"", ItemType:[Not supported], ValueValue:[Not supported], LegacyIAccessibleChildId:[not supported], LegacyIAccessibleName:[not supported], LegacyIAccessibleValue:[not supported], LegacyIAccessibleDescription:[not supported], LegacyIAccessibleRole:[not supported], LegacyIAccessibleState:[not supported], LegacyIAccessibleHelp:[not supported]