IUIAutomation MS framework automate chrome, FF, IE, ....

junkew · August 30, 2013

@ValeryVal:

* Try to increase sleep to a higher value (no proper way yet for synchronising, can be document is not yet fully loaded

* add a findThemAll($oDocument,$treescope_subtree) just after the consolewrite

it will dump more details of the tree thats getting searched

sleep(5000)
consolewrite("Action 7 retrieve document after clicking a hyperlink" & @CRLF)
    findThemAll($oDocument,$treescope_subtree)

    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Forum", $treescope_subtree)
    if not isobj($oForumLink) Then
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oForumLink,"invoke")
    sleep(3000)

@Lyr: Search for internet java accessibility, you should turn accessibility on for java programs and then it should in general work also with Awt controls.

Edited May 5, 2019 by junkew

junkew · August 30, 2013

Seems to be more complicated for Java applications (have their own api)

http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm

I see no reason why this api cannot be catched/used from AutoIT but will lead to a different thread (far in the future;-)).

Although accessible with javamonkey there seems not to be a bridge between MSAA-UIA to Java Accessibility bridge

At least all documentation I could find for future

Link for java (so also AWT) accessibility installation is here http://docs.oracle.com/javase/accessbridge/2.0.2/setup.htm

I think by default its nowadays installed when you install JRE/JDK 7

C:Program Files (x86)Javajre7lib

edit then accessibility.properties
uncomment
#assistive_technologies=com.sun.java.accessibility.AccessBridge
#screen_magnifier_present=true
Do this
http://docs.oracle.com/javase/7/docs/technotes/guides/access/enable_and_test.html#enabling_jab

download: JDK 7 and JavaFX Demos and Samples
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html
Windows x86 14.89 MB jdk-7u25-windows-i586-demos.zip Windows x64 14.98 MB jdk-7u25-windows-x64-demos.zip
Install the Java Ferret
http://docs.oracle.com/javase/accessbridge/2.0.2/javaferret.htm
The Java Access Bridge testing tools Java Monkey and Java Ferret are not shipped with the JDK. These tools are included in Java Access Bridge 2.0.2.
doubleclick on the
jdk-7u25-windows-x64-demosjdk1.7.0_25demojfcSwingSet2SwingSet2.Jar
Use the JavaMonkey program to investigate the SwingSet2 application

edit: added details on java demos

Edited August 30, 2013 by junkew

junkew · September 1, 2013

added checks if accessibility is on in example 6

consolewrite("Action 4a" & @CRLF)
$oChromeAddressBar=_UIA_getObjectByFindAll($oChromeToolbar, $cAddressBarByName  , $treescope_subtree) ;works in chrome 29
if not isobj($oChromeAddressbar) Then
findThemAll($oChromeToolbar,$treescope_subtree)
EndIf
 
$t=stringsplit(_UIA_getPropertyValue($oChromeAddressBar, $UIA_BoundingRectanglePropertyId),";")
_DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
_UIA_action($oChromeAddressBar,"leftclick")
 
 
consolewrite("Action 4b" & @CRLF)
;~ give some time to open website
sleep(2000)
$oDocument=_UIA_getFirstObjectOfElement($oChrome,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
if not isobj($oDocument) Then
findThemAll($oChrome,$treescope_subtree)
Else
$t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
_DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
EndIf
 
sleep(500)
 
consolewrite("Action 4c retrieve document after clicking a hyperlink" & @CRLF)
$oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=On", $treescope_subtree)
if not isobj($oForumLink) Then
consolewrite("*** Scripting will fail as accessibility is off ****")
MsgBox(4096, "Accessibility warning", "Accessibility is turned off, put it on by clicking on Off after Global accessibility mode", 10)
findThemAll($oDocument,$treescope_subtree)
EndIf
 
consolewrite("Action 4d" & @CRLF)
;~ $oChromeNewTab= _UIA_getFirstObjectOfElement($oChromeTabs,"controltype:=" & $UIA_ButtonControlTypeId, $treescope_subtree)
$oChromeNewTab= _UIA_getObjectByFindAll($oChromeTabs, $cChromeNewTabByName,$treescope_subtree)
if not isobj($oChromeNewTab) Then
findThemAll($oChromeTabs,$treescope_subtree)
EndIf
_UIA_action($oChromeNewtab,"leftclick")
sleep(500)

edit: multiple times reedited to get code in Edited September 1, 2013 by junkew

ValeryVal · September 2, 2013

@ValeryVal:
* Try to increase sleep to a higher value (no proper way yet for synchronising, can be document is not yet fully loaded
* add a findThemAll($oDocument,$treescope_subtree) just after the consolewrite
it will dump more details of the tree thats getting searched
sleep(5000)
consolewrite("Action 7 retrieve document after clicking a hyperlink" & @CRLF)
    findThemAll($oDocument,$treescope_subtree)

    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Forum", $treescope_subtree)
    if not isobj($oForumLink) Then
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oForumLink,"invoke")
    sleep(3000)
@Lyr: Search for internet java accessibility, you should turn accessibility on for java programs and then it should in general work also with Awt controls.

You are right. I've here slow internet connection. After your amendment the script had some progress. Now script reports about this:

...
Action 7 retrieve document after clicking a hyperlink
Title is: <>    Class   := <>   controltype:= <50030>   
Title is: <>    Class   := <>   controltype:= <50030>   
"D:\Program Files\AutoIt3\Examples\IUIAutomation\UIAWrappers.au3" (424) : ==> Variable must be of type "Object".:
$obj.getCurrentPattern($PatternId, $pPattern)
$obj^ ERROR
->11:11:08 AutoIT3.exe ended.rc:1
>Exit code: 1    Time: 23.379

Edited September 2, 2013 by ValeryVal

junkew · September 2, 2013

99.99% sure you do not have turned accessibility on in chrome (close chrome before you start the script or do below)

type
chrome://accessibility

in the addressbar to see if accessibility is on

The error you get actually means that the step before the webpage document is determined but as no details are retrievable (accessibility is off) the html elements can not be found.

I will handle those errors in update of uiawrappers but then script will not break just set @error value.

Edited September 2, 2013 by junkew

ValeryVal · September 2, 2013

99.99% sure you do not have turned accessibility on in chrome (close chrome before you start the script or do below)
type
chrome://accessibility
in the addressbar to see if accessibility is on
The error you get actually means that the step before the webpage document is determined but as no details are retrievable (accessibility is off) the html elements can not be found.
I will handle those errors in update of uiawrappers but then script will not break just set @error value.

I have always chrome closed before start the script.

Now I decided to skip Action 7 and script goes directly to www.autoitscript.com/forum/

;consolewrite("Action 7 retrieve document after clicking a hyperlink" & @CRLF)
;    findThemAll($oDocument,$treescope_subtree)

;    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Home", $treescope_subtree)
;    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Home", $treescope_subtree)
;    if not isobj($oForumLink) Then
;        consolewrite("$oForumLink is not obj" & @CRLF)
;        findThemAll($oDocument,$treescope_subtree)
;    EndIf
;    _UIA_action($oForumLink,"invoke")
;    sleep(3000)


consolewrite("Action 8 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oChrome,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
    if not isobj($oDocument) Then
        consolewrite("$oDocument is not obj" & @CRLF)
        findThemAll($oChrome,$treescope_subtree)
    Else
        $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
        _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    EndIf

;~ Now we get the searchfield

    $oEdtSearchForum=_UIA_getObjectByFindAll($oDocument,"name:=Search...", $treescope_subtree)
    if not isobj($oEdtSearchForum) Then
        consolewrite("$oEdtSearchForum is not obj" & @CRLF)
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oEdtSearchForum,"focus")
    _UIA_action($oEdtSearchForum,"setvalue using keys","Chrome can easy be automated with ui automation") ; {ENTER}")
    sleep(500)

;~ Exit
;~ Now we press the button, see relative syntax used as the button seems not to have a name its just 2 objects further then search field
    $oBtnSearch=_UIA_getObjectByFindAll($oDocument,"name:=Search... indexrelative:=2", $treescope_subtree)
    if not isobj($oBtnSearch) Then
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
    _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    sleep(1000)
    _UIA_action($oBtnSearch,"invoke")
    sleep(2000)

;~ consolewrite("Action 9 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oChrome,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
    if not isobj($oDocument) Then
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    $oHyperlink=_UIA_getObjectByFindAll($oDocument,"name:=controlsend doesn't work", $treescope_subtree)
    if not isobj($oBtnSearch) Then
        findThemAll($oDocument,$treescope_subtree)
    EndIf
    sleep(1000)
    _UIA_action($oHyperlink,"invoke")
    sleep(2000)

EndIf

exit

The result is the following:

Action 6
Action 8 first refresh the document control
Search...Found the Name is: <Search...> Class   := <>   controltype:= <50004>   
Relative object: name:=Search...
2
Relative position usedSearch...Found the Name is: <>    Class   := <>   controltype:= <50000>   
controlsend doesn't workFound the Name is: <controlsend doesn't work>   Class   := <>   controltype:= <50005>   
+>17:15:23 AutoIT3.exe ended.rc:0
>Exit code: 0    Time: 32.649

And chrome exposes page with URL:

Thank you.

junkew · September 2, 2013

Which OS version are you on?

Which Chrome version? Make sure its 29 and not 28 (I think 28 is not exposing name property allways correct)

Can you run the script from the first post and hover your mouse to the find hyperlink to see what is exposed.

If you have inspect.exe can you tell what you see when you hover over the html elements

I will enhance findThemAll routine to print more information on the objects and related siblings

As step 8 seems to be doing well and you get the link as you show accessibility is indeed turned on and somehow the name of the object is different on your system. As you click on the hyperlink in the html page in step 8 the accessibility and controlling works. Somehow the property value is different on your system.

Keep example 6 as is, no need to comment out, step 7 should work fine, add steps 4a,b,c,d (recopy example 6)

ValeryVal · September 3, 2013

Which OS version are you on?

Which Chrome version? Make sure its 29 and not 28 (I think 28 is not exposing name property allways correct)

Can you run the script from the first post and hover your mouse to the find hyperlink to see what is exposed.

If you have inspect.exe can you tell what you see when you hover over the html elements

I will enhance findThemAll routine to print more information on the objects and related siblings

As step 8 seems to be doing well and you get the link as you show accessibility is indeed turned on and somehow the name of the object is different on your system. As you click on the hyperlink in the html page in step 8 the accessibility and controlling works. Somehow the property value is different on your system.

Keep example 6 as is, no need to comment out, step 7 should work fine, add steps 4a,b,c,d (recopy example 6)

>Which OS version are you on?

Win 7 Pro.

>Which Chrome version?

chrome://chrome/ shows 29.0.1547.62 m (the last one)

>Keep example 6 as is, no need to comment out, step 7 should work fine,

Script reports (report has russian letter):

>Running:(3.3.9.2):D:\Program Files\AutoIt3\autoit3.exe "D:\Program Files\AutoIt3\Examples\IUIAutomation\Example6.au3"    
Action 1
Action 2
Адресная строка и строка поискаFound the Name is: <Адресная строка и строка поиска>   Class   := <>   controltype:= <50025>   
Action 3
Action 4
Новая вкладкаFound the Name is: <Новая вкладка> Class   := <>   controltype:= <50000>   
Action 5
Адресная строка и строка поискаFound the Name is: <Адресная строка и строка поиска>   Class   := <>   controltype:= <50025>   
Action 6
Action 7 retrieve document after clicking a hyperlink
Title is: <>    Class   := <>   controltype:= <50030>   
$oForumLink is not obj
Title is: <>    Class   := <>   controltype:= <50030>   
"D:\Program Files\AutoIt3\Examples\IUIAutomation\UIAWrappers.au3" (424) : ==> Variable must be of type "Object".:
$obj.getCurrentPattern($PatternId, $pPattern)
$obj^ ERROR
->08:18:02 AutoIT3.exe ended.rc:1
>Exit code: 1    Time: 22.615

junkew · September 3, 2013

Debugging 1

Go to first post

use that example and comment this out

;~if _ispressed(01) Then

;~getelementinfo()

;~endif

Hover to the element and press <Ctrl><M> when you hover over the link Forum on homepage of autoit

http://www.autoitscript.com/site/

What is the output in the textbox regarding all properties?

Debugging 2

for the tab you are looking at press show accessibility tree

Should show something like

ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(668, 15) size=(48, 51) index_in_parent=5 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Forum' name='Forum' value='http://www.autoitscript.com/forum/' FOCUSABLE LINKED TRAVERSED display:block tag:a location=(668, 15) size=(38, 51) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Forum' name='Forum' READONLY LINKED TRAVERSED display:block location=(670, 22) size=(35, 16) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0

What is the output of the tree?

Based on debugging 2 you could try to find the first with a name

and use it in combination with indexrelative property

syntax example

"name:=Search... indexrelative:=2"

Edited September 3, 2013 by junkew

ValeryVal · September 4, 2013

Debugging 1
Go to first post
use that example and comment this out
;~if _ispressed(01) Then
;~getelementinfo()
;~endif
Hover to the element and press <Ctrl><M> when you hover over the link Forum on homepage of autoit
http://www.autoitscript.com/site/

What is the output in the textbox regarding all properties?

Debugging 2
for the tab you are looking at press show accessibility tree
Should show something like
ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(668, 15) size=(48, 51) index_in_parent=5 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Forum' name='Forum' value='http://www.autoitscript.com/forum/' FOCUSABLE LINKED TRAVERSED display:block tag:a location=(668, 15) size=(38, 51) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Forum' name='Forum' READONLY LINKED TRAVERSED display:block location=(670, 22) size=(35, 16) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
What is the output of the tree?

Based on debugging 2 you could try to find the first with a name
and use it in combination with indexrelative property

syntax example
"name:=Search... indexrelative:=2"

Thank your for support in chrome study.

Debugging 1

>What is the output in the textbox regarding all properties?

Mouse position is retrieved 1164-340
At least we have an element [Forum][]
Having the following values for all properties: 
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=
UIA_BoundingRectanglePropertyId :=1139;331;36;17
UIA_ClassNamePropertyId :=
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50004
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=False
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=False
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=True
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Forum
UIA_LegacyIAccessibleRolePropertyId :=42
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=12582976
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=????
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Forum
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=2444
UIA_ProviderDescriptionPropertyId :=[pid:2444,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Debugging 2

>What is the output of the tree?

The tab with URL

chrome://accessibility/#

reports about AutoItScript - AutoItScript Website:

ROLE_SYSTEM_DOCUMENT name='AutoItScript - AutoItScript Website' name='AutoItScript - AutoItScript Website' value='http://www.autoitscript.com/site/' READONLY FOCUSABLE tag:#document location=(0, 0) size=(1106, 1112) index_in_parent=0 n_relations=0 n_characters=30 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoItScript' name='AutoItScript' value='http://www.autoitscript.com/site' FOCUSABLE LINKED display:inline tag:a location=(73, 0) size=(210, 72) index_in_parent=0 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoItScript' name='AutoItScript' value='http://aut1.autoit-cdn.com/site/wp-content/themes/TheCorporation/images/logo@2x.png' READONLY LINKED display:block tag:img location=(73, 0) size=(210, 72) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LIST READONLY display:block tag:ul location=(336, 17) size=(520, 56) index_in_parent=1 n_relations=0 n_characters=9 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(336, 17) size=(49, 56) index_in_parent=0 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Home' name='Home' value='http://www.autoitscript.com/site/' FOCUSABLE LINKED TRAVERSED display:block tag:a location=(336, 17) size=(37, 56) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Home' name='Home' READONLY LINKED TRAVERSED display:block location=(339, 26) size=(34, 17) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(385, 17) size=(57, 56) index_in_parent=1 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='AutoIt »' name='AutoIt »' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(385, 17) size=(45, 56) index_in_parent=0 n_relations=0 n_characters=8 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='AutoIt' name='AutoIt' READONLY LINKED display:block location=(388, 26) size=(33, 17) index_in_parent=0 n_relations=0 n_characters=6 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name=' »' name=' »' READONLY LINKED display:inline location=(421, 26) size=(9, 17) index_in_parent=1 n_relations=0 n_characters=2 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(442, 17) size=(93, 56) index_in_parent=2 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='AutoIt Editor »' name='AutoIt Editor »' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(442, 17) size=(81, 56) index_in_parent=0 n_relations=0 n_characters=15 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='AutoIt Editor' name='AutoIt Editor' READONLY LINKED display:block location=(445, 26) size=(69, 17) index_in_parent=0 n_relations=0 n_characters=13 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name=' »' name=' »' READONLY LINKED display:inline location=(514, 26) size=(9, 17) index_in_parent=1 n_relations=0 n_characters=2 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(535, 17) size=(90, 56) index_in_parent=3 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='AutoIt Tools »' name='AutoIt Tools »' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(535, 17) size=(78, 56) index_in_parent=0 n_relations=0 n_characters=14 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='AutoIt Tools' name='AutoIt Tools' READONLY LINKED display:block location=(538, 26) size=(66, 17) index_in_parent=0 n_relations=0 n_characters=12 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name=' »' name=' »' READONLY LINKED display:inline location=(604, 26) size=(9, 17) index_in_parent=1 n_relations=0 n_characters=2 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(625, 17) size=(44, 56) index_in_parent=4 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Code' name='Code' value='http://www.autoitscript.com/site/code/' FOCUSABLE LINKED display:block tag:a location=(625, 17) size=(32, 56) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Code' name='Code' READONLY LINKED display:block location=(628, 26) size=(29, 17) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(669, 17) size=(51, 56) index_in_parent=5 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Forum' name='Forum' value='http://www.autoitscript.com/forum/' FOCUSABLE LINKED TRAVERSED display:block tag:a location=(669, 17) size=(39, 56) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Forum' name='Forum' READONLY LINKED TRAVERSED display:block location=(672, 26) size=(36, 17) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(720, 17) size=(38, 56) index_in_parent=6 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Wiki' name='Wiki' value='http://www.autoitscript.com/wiki' FOCUSABLE LINKED display:block tag:a location=(720, 17) size=(26, 56) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Wiki' name='Wiki' READONLY LINKED display:block location=(723, 26) size=(23, 17) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(758, 17) size=(40, 56) index_in_parent=7 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Blog' name='Blog' value='http://www.autoitscript.com/site/blog/' FOCUSABLE LINKED display:block tag:a location=(758, 17) size=(28, 56) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Blog' name='Blog' READONLY LINKED display:block location=(761, 26) size=(25, 17) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LISTITEM READONLY display:list-item tag:li location=(798, 17) size=(58, 56) index_in_parent=8 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='Contact' name='Contact' value='http://www.autoitscript.com/site/contact-us/' FOCUSABLE LINKED display:block tag:a location=(798, 17) size=(46, 56) index_in_parent=0 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='Contact' name='Contact' READONLY LINKED display:block location=(801, 26) size=(43, 17) index_in_parent=0 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='search' name='search' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(950, 48) size=(83, 25) index_in_parent=2 n_relations=0 n_characters=6 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='search' name='search' READONLY LINKED display:block location=(981, 52) size=(33, 15) index_in_parent=0 n_relations=0 n_characters=6 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING name='autoit' name='autoit' READONLY display:block tag:h2 level:2 role_name='h2' location=(147, 109) size=(790, 83) index_in_parent=3 n_relations=0 n_characters=6 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='autoit' name='autoit' READONLY display:block location=(147, 109) size=(157, 69) index_in_parent=0 n_relations=0 n_characters=6 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(157, 192) size=(780, 35) index_in_parent=4 n_relations=0 n_characters=33 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='AUTOMATION AND SCRIPTING LANGUAGE' name='AUTOMATION AND SCRIPTING LANGUAGE' READONLY display:block location=(157, 193) size=(314, 19) index_in_parent=0 n_relations=0 n_characters=33 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(157, 227) size=(415, 143) index_in_parent=5 n_relations=0 n_characters=450 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying “runtimes”...' name='AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks in a way not possible or reliable with other languages (e.g. VBScript and SendKeys). AutoIt is also very small, self-contained and will run on all versions of Windows out-of-the-box with no annoying “runtimes”...' READONLY display:block location=(157, 227) size=(415, 132) index_in_parent=0 n_relations=0 n_characters=450 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='read more' name='read more' value='http://www.autoitscript.com/site/autoit/' FOCUSABLE LINKED display:block tag:a description='AutoIt' location=(427, 388) size=(145, 45) index_in_parent=6 n_relations=0 n_characters=9 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='read more' name='read more' READONLY LINKED display:block location=(467, 397) size=(79, 19) index_in_parent=0 n_relations=0 n_characters=9 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoIt' name='AutoIt' value='http://www.autoitscript.com/site/autoit/' FOCUSABLE LINKED display:inline tag:a description='AutoIt' location=(607, 227) size=(330, 220) index_in_parent=7 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoIt' name='AutoIt' value='http://aut1.autoit-cdn.com/site/wp-content/themes/TheCorporation/images/custom/featured-autoit.jpg' READONLY LINKED display:block tag:img location=(607, 227) size=(330, 220) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING name='autoit tools' name='autoit tools' READONLY display:block tag:h2 level:2 role_name='h2' location=(147, 109) size=(790, 83) index_in_parent=8 n_relations=0 n_characters=12 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='autoit tools' name='autoit tools' READONLY display:block location=(147, 109) size=(299, 69) index_in_parent=0 n_relations=0 n_characters=12 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(157, 192) size=(780, 35) index_in_parent=9 n_relations=0 n_characters=30 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='DEPLOYMENT SOLUTIONS AND TOOLS' name='DEPLOYMENT SOLUTIONS AND TOOLS' READONLY display:block location=(157, 193) size=(287, 19) index_in_parent=0 n_relations=0 n_characters=30 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(157, 227) size=(415, 143) index_in_parent=10 n_relations=0 n_characters=466 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Welcome to the AutoIt Tools project. AutoIt Tools are  a suite of tools intended to help out in the follow areas:Build scripting and configuration Login scripts General IT administrationWhile AutoIt can be used to achieve these goals, many IT administrators are reluctant to install and learn a full scripting language when they only require a specific function. To this end AutoIt Tools will consist of small and self-contained executables that can be used for a...' name='Welcome to the AutoIt Tools project. AutoIt Tools are  a suite of tools intended to help out in the follow areas:Build scripting and configuration Login scripts General IT administrationWhile AutoIt can be used to achieve these goals, many IT administrators are reluctant to install and learn a full scripting language when they only require a specific function. To this end AutoIt Tools will consist of small and self-contained executables that can be used for a...' READONLY display:block location=(157, 227) size=(413, 132) index_in_parent=0 n_relations=0 n_characters=466 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='read more' name='read more' value='http://www.autoitscript.com/site/autoit-tools/' FOCUSABLE LINKED display:block tag:a description='AutoIt Tools' location=(427, 388) size=(145, 45) index_in_parent=11 n_relations=0 n_characters=9 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='read more' name='read more' READONLY LINKED display:block location=(467, 397) size=(79, 19) index_in_parent=0 n_relations=0 n_characters=9 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoIt Tools' name='AutoIt Tools' value='http://www.autoitscript.com/site/autoit-tools/' FOCUSABLE LINKED display:inline tag:a description='AutoIt Tools' location=(607, 227) size=(330, 220) index_in_parent=12 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoIt Tools' name='AutoIt Tools' value='http://aut1.autoit-cdn.com/site/wp-content/themes/TheCorporation/images/custom/featured-autoit-tools.png' READONLY LINKED display:block tag:img location=(607, 227) size=(330, 220) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='Previous' name='Previous' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(-9926, 246) size=(10055, 69) index_in_parent=13 n_relations=0 n_characters=8 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Previous' name='Previous' READONLY LINKED display:block location=(-9926, 247) size=(52, 18) index_in_parent=0 n_relations=0 n_characters=8 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='Next' name='Next' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:block tag:a location=(-9022, 246) size=(10055, 69) index_in_parent=14 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Next' name='Next' READONLY LINKED display:block location=(-9022, 247) size=(29, 18) index_in_parent=0 n_relations=0 n_characters=4 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoIt' name='AutoIt' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:inline tag:a location=(156, 511) size=(82, 82) index_in_parent=15 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoIt' name='AutoIt' value='http://aut1.autoit-cdn.com/site/wp-content/uploads/et_temp/featured-autoit-10402_72x72.jpg' READONLY LINKED display:block tag:img location=(156, 511) size=(82, 82) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='GImageX' name='GImageX' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:inline tag:a location=(268, 511) size=(82, 82) index_in_parent=16 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='GImageX' name='GImageX' value='http://aut1.autoit-cdn.com/site/wp-content/uploads/et_temp/featured-gimagex-8417_72x72.png' READONLY LINKED display:block tag:img location=(268, 511) size=(82, 82) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoIt Script Editor' name='AutoIt Script Editor' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:inline tag:a location=(380, 511) size=(82, 82) index_in_parent=17 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoIt Script Editor' name='AutoIt Script Editor' value='http://aut1.autoit-cdn.com/site/wp-content/uploads/et_temp/featured-autoit-editor-11868_72x72.png' READONLY LINKED display:block tag:img location=(380, 511) size=(82, 82) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK name='AutoIt Tools' name='AutoIt Tools' value='http://www.autoitscript.com/site/#' FOCUSABLE LINKED display:inline tag:a location=(492, 511) size=(82, 82) index_in_parent=18 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_GRAPHIC name='AutoIt Tools' name='AutoIt Tools' value='http://aut1.autoit-cdn.com/site/wp-content/uploads/et_temp/featured-autoit-tools-3505_72x72.png' READONLY LINKED display:block tag:img location=(492, 511) size=(82, 82) index_in_parent=0 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING name='About' name='About' READONLY display:block tag:h3 level:3 role_name='h3' location=(123, 669) size=(530, 35) index_in_parent=19 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='About' name='About' READONLY display:block location=(171, 672) size=(49, 21) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(123, 722) size=(530, 48) index_in_parent=20 n_relations=0 n_characters=140 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Welcome to the AutoIt website. Home of the AutoIt scripting language and other useful deployment related tools, documentation and utilities.' name='Welcome to the AutoIt website. Home of the AutoIt scripting language and other useful deployment related tools, documentation and utilities.' READONLY display:block location=(123, 722) size=(518, 37) index_in_parent=0 n_relations=0 n_characters=140 caret_offset=0 n_selections=0
    ROLE_SYSTEM_LINK value='http://www.autoitscript.com/site/feed/' FOCUSABLE LINKED display:inline tag:a location=(730, 691) size=(22, 23) index_in_parent=21 n_relations=0 n_characters=0 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING name='From the Blog' name='From the Blog' READONLY display:block tag:h3 level:3 role_name='h3' location=(729, 691) size=(254, 35) index_in_parent=22 n_relations=0 n_characters=13 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='From the Blog' name='From the Blog' READONLY display:block location=(764, 691) size=(114, 21) index_in_parent=0 n_relations=0 n_characters=13 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING READONLY display:block tag:h4 level:4 role_name='h4' location=(733, 728) size=(250, 18) index_in_parent=23 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='AutoIt v3.3.8.1 Released' name='AutoIt v3.3.8.1 Released' value='http://www.autoitscript.com/site/autoit-news/autoit-v3-3-8-1-released/' FOCUSABLE LINKED display:inline tag:a description='Permanent Link to AutoIt v3.3.8.1 Released' location=(752, 728) size=(132, 15) index_in_parent=0 n_relations=0 n_characters=24 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='AutoIt v3.3.8.1 Released' name='AutoIt v3.3.8.1 Released' READONLY LINKED display:inline location=(752, 728) size=(132, 15) index_in_parent=0 n_relations=0 n_characters=24 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(733, 746) size=(250, 26) index_in_parent=24 n_relations=0 n_characters=27 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Posted by ' name='Posted by ' READONLY display:block location=(733, 746) size=(55, 15) index_in_parent=0 n_relations=0 n_characters=10 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Jonathan Bennett' name='Jonathan Bennett' value='http://www.autoitscript.com/site/author/admin/' FOCUSABLE LINKED display:inline tag:a description='Posts by Jonathan Bennett' location=(788, 746) size=(91, 15) index_in_parent=1 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Jonathan Bennett' name='Jonathan Bennett' READONLY LINKED display:inline location=(788, 746) size=(91, 15) index_in_parent=0 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' on Jan 29, 2012' name=' on Jan 29, 2012' READONLY display:block location=(879, 746) size=(81, 15) index_in_parent=2 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING READONLY display:block tag:h4 level:4 role_name='h4' location=(733, 774) size=(250, 18) index_in_parent=25 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='AutoIt v3.3.8.0 Released' name='AutoIt v3.3.8.0 Released' value='http://www.autoitscript.com/site/autoit-news/autoit-v3-3-8-0-released/' FOCUSABLE LINKED display:inline tag:a description='Permanent Link to AutoIt v3.3.8.0 Released' location=(752, 774) size=(132, 15) index_in_parent=0 n_relations=0 n_characters=24 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='AutoIt v3.3.8.0 Released' name='AutoIt v3.3.8.0 Released' READONLY LINKED display:inline location=(752, 774) size=(132, 15) index_in_parent=0 n_relations=0 n_characters=24 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(733, 792) size=(250, 26) index_in_parent=26 n_relations=0 n_characters=27 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Posted by ' name='Posted by ' READONLY display:block location=(733, 792) size=(55, 15) index_in_parent=0 n_relations=0 n_characters=10 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Jonathan Bennett' name='Jonathan Bennett' value='http://www.autoitscript.com/site/author/admin/' FOCUSABLE LINKED display:inline tag:a description='Posts by Jonathan Bennett' location=(788, 792) size=(91, 15) index_in_parent=1 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Jonathan Bennett' name='Jonathan Bennett' READONLY LINKED display:inline location=(788, 792) size=(91, 15) index_in_parent=0 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' on Dec 23, 2011' name=' on Dec 23, 2011' READONLY display:block location=(879, 792) size=(84, 15) index_in_parent=2 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
    IA2_ROLE_HEADING READONLY display:block tag:h4 level:4 role_name='h4' location=(733, 820) size=(250, 18) index_in_parent=27 n_relations=0 n_characters=1 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='AutoIt Consulting Deployment Blog Posts' name='AutoIt Consulting Deployment Blog Posts' value='http://www.autoitscript.com/site/deployment/autoit-consulting-deployment-blog-posts/' FOCUSABLE LINKED display:inline tag:a description='Permanent Link to AutoIt Consulting Deployment Blog Posts' location=(752, 820) size=(225, 15) index_in_parent=0 n_relations=0 n_characters=39 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='AutoIt Consulting Deployment Blog Posts' name='AutoIt Consulting Deployment Blog Posts' READONLY LINKED display:inline location=(752, 820) size=(225, 15) index_in_parent=0 n_relations=0 n_characters=39 caret_offset=0 n_selections=0
    IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(733, 838) size=(250, 26) index_in_parent=28 n_relations=0 n_characters=27 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name='Posted by ' name='Posted by ' READONLY display:block location=(733, 838) size=(55, 15) index_in_parent=0 n_relations=0 n_characters=10 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Jonathan Bennett' name='Jonathan Bennett' value='http://www.autoitscript.com/site/author/admin/' FOCUSABLE LINKED display:inline tag:a description='Posts by Jonathan Bennett' location=(788, 838) size=(91, 15) index_in_parent=1 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Jonathan Bennett' name='Jonathan Bennett' READONLY LINKED display:inline location=(788, 838) size=(91, 15) index_in_parent=0 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' on May 10, 2011' name=' on May 10, 2011' READONLY display:block location=(879, 838) size=(87, 15) index_in_parent=2 n_relations=0 n_characters=16 caret_offset=0 n_selections=0
    IA2_ROLE_SECTION READONLY role_name='div' location=(73, 1031) size=(960, 29) index_in_parent=29 n_relations=0 n_characters=14 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Company Details' name='Company Details' value='http://www.autoitconsulting.com/site/about/company-details/' FOCUSABLE LINKED display:inline tag:a location=(73, 1032) size=(101, 18) index_in_parent=0 n_relations=0 n_characters=15 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Company Details' name='Company Details' READONLY LINKED display:inline location=(73, 1032) size=(101, 18) index_in_parent=0 n_relations=0 n_characters=15 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' - ' name=' - ' READONLY display:block location=(174, 1032) size=(10, 18) index_in_parent=1 n_relations=0 n_characters=3 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Terms' name='Terms' value='http://www.autoitconsulting.com/site/terms-of-service/' FOCUSABLE LINKED display:inline tag:a location=(184, 1032) size=(37, 18) index_in_parent=2 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Terms' name='Terms' READONLY LINKED display:inline location=(184, 1032) size=(37, 18) index_in_parent=0 n_relations=0 n_characters=5 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' - ' name=' - ' READONLY display:block location=(221, 1032) size=(10, 18) index_in_parent=3 n_relations=0 n_characters=3 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Privacy' name='Privacy' value='http://www.autoitconsulting.com/site/privacy-policy/' FOCUSABLE LINKED display:inline tag:a location=(231, 1032) size=(43, 18) index_in_parent=4 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Privacy' name='Privacy' READONLY LINKED display:inline location=(231, 1032) size=(43, 18) index_in_parent=0 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
        ROLE_SYSTEM_TEXT name=' - ' name=' - ' READONLY display:block location=(274, 1032) size=(10, 18) index_in_parent=5 n_relations=0 n_characters=3 caret_offset=0 n_selections=0
        ROLE_SYSTEM_LINK name='Cookies' name='Cookies' value='http://www.autoitconsulting.com/site/cookie-policy/' FOCUSABLE LINKED display:inline tag:a location=(284, 1032) size=(47, 18) index_in_parent=6 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='Cookies' name='Cookies' READONLY LINKED display:inline location=(284, 1032) size=(47, 18) index_in_parent=0 n_relations=0 n_characters=7 caret_offset=0 n_selections=0
        IA2_ROLE_PARAGRAPH display:block tag:p role_name='P' location=(711, 1031) size=(322, 29) index_in_parent=7 n_relations=0 n_characters=31 caret_offset=0 n_selections=0
            ROLE_SYSTEM_TEXT name='©1999-2012 Jonathan Bennett & ' name='©1999-2012 Jonathan Bennett & ' READONLY display:block location=(711, 1031) size=(194, 18) index_in_parent=0 n_relations=0 n_characters=30 caret_offset=0 n_selections=0
            ROLE_SYSTEM_LINK name='AutoIt Consulting Ltd' name='AutoIt Consulting Ltd' value='http://www.autoitconsulting.com/' FOCUSABLE LINKED display:inline tag:a location=(905, 1031) size=(128, 18) index_in_parent=1 n_relations=0 n_characters=21 caret_offset=0 n_selections=0
                ROLE_SYSTEM_TEXT name='AutoIt Consulting Ltd' name='AutoIt Consulting Ltd' READONLY LINKED display:inline location=(905, 1031) size=(128, 18) index_in_parent=0 n_relations=0 n_characters=21 caret_offset=0 n_selections=0

Based on debugging 2 you could try to find the first with a name
and use it in combination with indexrelative property

What name should I search?

junkew · September 4, 2013

Debugging 1 and 2 shows at least the name:=Forum construct should work.

Seems somehow that we are searching belong the wrong parent or having the incorrect document uia element

You could rewrite to:

$oForumLink=_UIA_getObjectByFindAll($oChrome,"name:=Forum", 
$treescope_subtree)

This way it starts searching higher in the hierarchy of elements so less efficient but most likely will work

Lyr · September 4, 2013

Since you took some notes about Java, here is some other reference that you may be interested: page-5'?do=embed' frameborder='0' data-embedContent>>

It's old (2009) but the JAB from then seems to be 2.0.1, not that far from 2.0.2. It misses the dll source, but the Java.au3 may get you started.

junkew · September 4, 2013

saw that link but thats using an external dll. I have the basics working on my PC with only AutoIT code but some trouble with 32 bits and 64 bits calling conventions in dll. Will post the basics in a different thread

oceanwaves · September 10, 2013

Thanks, I think it's very helpful. I read some article releted with UI automation API on the microsoft website, but was still unknowing how to embed them to Autoit before you topic. It's awesome, how do you do it!

Appreciate your UDF.

BTW, http://msdn.microsoft.com/en-us/library/jj160544.aspx this site provide a tools can test many element on the GUI, although I don't clear know the detail information about testing failed message. Because I try it on the notepad, some test case will fail.

junkew · September 15, 2013

Example 7 The chrome example modified to a firefox example.

Very powerfull is that in CHROME and FF you can use the same search mechanism to come to the HTML document window as from that point code is 99% equal to click on links, set values etc.

Small changes compared to the chrome example

1. Renamed CHROME to FF

2. Some constants defined differently to map with FF names

const $cToolbarByName = "name:=Navigation Toolbar"
const $cAddressBarByName = "name:=Search or enter address"
const $cFFNewTabByName="name:=Open a new tab"

3. Start firefox different executable offcourse

4. Removed steps 4a thru 4d as accessibility tree within firefox is thru dominspector of the tools menu

5. Not clicking a new tab in step 4

logic to be added to uiawrappers to have something like indexlast, FF is having multiple documents active invisible

make sure all tabs in firefox are closed

6. In step 7 not invoke but leftclick (FF is not supporting invoke)

_UIA_action($oForumLink,"leftclick")

7. same in step 8

_UIA_action($oBtnSearch,"leftclick")

8. and in step 9

_UIA_action($oHyperlink,"leftclick")

unfortunately index for search is 4 indexes away instead of 2 (TODO: More flexible when not finding by name)

$oBtnSearch=_UIA_getObjectByFindAll($oDocument,"name:=Search... indexrelative:=4", $treescope_subtree)

And the full firefox example

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

;~ Make this language specific
const $cToolbarByName = "name:=Navigation Toolbar"
const $cAddressBarByName = "name:=Search or enter address"
const $cFFNewTabByName="name:=Open a new tab"

$strFFExeFolder="C:\Program Files (x86)\Mozilla Firefox\"
$strFFStartup=""
$strFFExe=$strFFExeFolder & "firefox.exe "

;~ Start chrome
if fileexists($strFFExe) Then
    if not processexists("firefox.exe") Then
        run($strFFExe & $strFFStartup,"", @SW_MAXIMIZE )
        ProcessWait("firefox.exe")
        ;~ Just to give some time to start
        sleep(10000)
    endif
Else
    consolewrite("No clue where to find firefox on your system, please start manually:" & @CRLF )
    consolewrite($strFFExe & $strFFStartup & @CRLF)
EndIf

;~ Find the FF window
$oFF=_UIA_getFirstObjectOfElement($oDesktop,"class:=MozillaWindowClass", $treescope_children)
if not isobj($oFF) Then
    _UIA_DumpThemAll($oDesktop,$treescope_children)
EndIf

;~ Make sure chrome is front window
$oFF.setfocus()

if isobj($oFF) Then
    consolewrite("Action 1" & @CRLF)

;~  get the chrome toolbar
;~  $oFFToolbar=_UIA_getFirstObjectOfElement($oFF,"controltype:=" & $UIA_ToolBarControlTypeId, $treescope_subtree)
    $oFFToolbar=_UIA_getFirstObjectOfElement($oFF,$cToolbarByName, $treescope_subtree)
    if not isobj($oFFToolbar) Then
        _UIA_DumpThemAll($oFF,$treescope_subtree)
    EndIf

consolewrite("Action 2" & @CRLF)
;~  get the addressbar
;~  $oFFAddressBar=_UIA_getFirstObjectOfElement($oFFToolbar,"class:=Chrome_OmniboxView", $treescope_children) ;worked in chrome 28
;~  $oFFAddressBar=_UIA_getFirstObjectOfElement($oFFToolbar,"controltype:=" & $UIA_EditControlTypeId , $treescope_subtree) ;works in chrome 29
;~  $oFFAddressBar=_UIA_getFirstObjectOfElement($oFFToolbar,"name:=Adres- en zoekbalk"  , $treescope_children) ;works in chrome 29
    $oFFAddressBar=_UIA_getObjectByFindAll($oFFToolbar, $cAddressBarByName  , $treescope_subtree) ;works in chrome 29
    if not isobj($oFFAddressbar) Then
        _UIA_DumpThemAll($oFFToolbar,$treescope_subtree)
    EndIf

;~  $oValueP=_UIA_getpattern($oFFAddressBar,$UIA_ValuePatternId)
;~  sleep(2000)

;~  get the value of the addressbar
;~  $myText=""
;~  $oValueP.CurrentValue($myText)
;~  consolewrite("address: " & $myText & @CRLF)

consolewrite("Action 3" & @CRLF)
;~ Get reference to the tabs
    $oFFTabs=_UIA_getFirstObjectOfElement($oFF,"controltype:=" & $UIA_TabControlTypeId, $treescope_subtree)
    if not isobj($oFFTabs) Then
        _UIA_DumpThemAll($oFF,$treescope_subtree)
    EndIf

;~ Lets open a new tab within chrome

consolewrite("Action 4" & @CRLF)
;~  $oFFNewTab= _UIA_getFirstObjectOfElement($oFFTabs,"controltype:=" & $UIA_ButtonControlTypeId, $treescope_subtree)
    $oFFNewTab= _UIA_getObjectByFindAll($oFFTabs, $cFFNewTabByName,$treescope_subtree)
    if not isobj($oFFNewTab) Then
        _UIA_DumpThemAll($oFFTabs,$treescope_subtree)
    EndIf
;~  _UIA_action($oFFNewtab,"leftclick")
;~  sleep(500)

consolewrite("Action 5" & @CRLF)
    $oFFAddressBar=_UIA_getObjectByFindAll($oFFToolbar, $cAddressBarByName  , $treescope_subtree) ;works in chrome 29

    if not isobj($oFFAddressbar) Then
        _UIA_DumpThemAll($oFFToolbar,$treescope_subtree)
    EndIf

    $t=stringsplit(_UIA_getPropertyValue($oFFAddressBar, $UIA_BoundingRectanglePropertyId),";")
    _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    _UIA_action($oFFAddressBar,"leftclick")
    _UIA_action($oFFAddressBar,"setvalue using keys","www.autoitscript.com/{ENTER}")

consolewrite("Action 6" & @CRLF)

;~  give some time to open website
    sleep(2000)
    $oDocument=_UIA_getFirstObjectOfElement($oFF,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oFF,$treescope_subtree)
    Else
        $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
        _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    EndIf

    sleep(500)

consolewrite("Action 7 retrieve document after clicking a hyperlink" & @CRLF)
    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Forum", $treescope_subtree)
;~ All document items
    if not isobj($oForumLink) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oForumLink,"leftclick")
    sleep(3000)

consolewrite("Action 8 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oFF,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oFF,$treescope_subtree)
    Else
        $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
        _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    EndIf

;~ Now we get the searchfield

    $oEdtSearchForum=_UIA_getObjectByFindAll($oDocument,"name:=Search...", $treescope_subtree)
    if not isobj($oEdtSearchForum) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oEdtSearchForum,"focus")
    _UIA_action($oEdtSearchForum,"setvalue using keys","Chrome can easy be automated with ui automation") ; {ENTER}")
    sleep(500)

;~ Exit
;~ Now we press the button, see relative syntax used as the button seems not to have a name its just 2 objects further then search field
    $oBtnSearch=_UIA_getObjectByFindAll($oDocument,"name:=Search...; indexrelative:=4", $treescope_subtree)
    if not isobj($oBtnSearch) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
    _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    sleep(1000)
    _UIA_action($oBtnSearch,"leftclick")
    sleep(2000)

;~ consolewrite("Action 9 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oFF,"controltype:=" & $UIA_DocumentControlTypeId , $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    $oHyperlink=_UIA_getObjectByFindAll($oDocument,"name:=controlsend doesn't work; controltype:=edit", $treescope_subtree)
    if not isobj($oBtnSearch) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    sleep(1000)
    _UIA_action($oHyperlink,"leftclick")
    sleep(2000)

EndIf

exit

Edited November 17, 2013 by junkew

junkew · September 19, 2013

Example 8 The other major browser Internet Explorer automated (made on Example 6 and 7)

This demonstrates that with almost identical code you can automate 3 major browsers

Small changes compared to the Fire fox example
1. Renamed FF to IE
2. Some constants defined differently to map with IE names

;~ Make this language specific
const $cToolbarByName = "classname:=ReBarWindow32" ;Navigation bar
const $cAddressBarByName = "name:=Adres en.*"
const $cIENewTabByName="name:=Nieuw.*"
const $cBrowser="classname:=IEFrame"
;~ const $cDocumentWindow="controltype:=" & $UIA_DocumentControlTypeId
const $cDocumentWindow="classname:=Internet Explorer_Server"
const $cBoardItem="name:=controlsend doesn't work"
const $cEdtSearch="name:=Search.*"
const $cSearchButton="name:=Search.*; indexrelative:=2"

The full source for this IE example

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

;~ Make this language specific
const $cToolbarByName = "classname:=ReBarWindow32" ;Navigation bar
const $cAddressBarByName = "name:=Adres en.*"
const $cIENewTabByName="name:=Nieuw.*"
const $cBrowser="classname:=IEFrame"
;~ const $cDocumentWindow="controltype:=" & $UIA_DocumentControlTypeId
const $cDocumentWindow="classname:=Internet Explorer_Server"
const $cBoardItem="name:=controlsend doesn't work"
const $cEdtSearch="name:=Search.*"
const $cSearchButton="name:=Search.*; indexrelative:=2"

$strIEExeFolder="C:\Program Files (x86)\Internet Explorer\"
$strIEStartup=""
$strIEExe=$strIEExeFolder & "iexplore.exe "

;~ Start chrome
if fileexists($strIEExe) Then
    if not processexists("iexplore.exe") Then
        run($strIEExe & $strIEStartup,"", @SW_MAXIMIZE )
        ProcessWait("iexplore.exe")
        ;~ Just to give some time to start
        sleep(10000)
    endif
Else
    consolewrite("No clue where to find IE on your system, please start manually:" & @CRLF )
    consolewrite($strIEExe & $strIEStartup & @CRLF)
EndIf

;~ Find the IE window
$oIE=_UIA_getFirstObjectOfElement($oDesktop, $cBrowser, $treescope_children)
if not isobj($oIE) Then
    _UIA_DumpThemAll($oDesktop,$treescope_children)
EndIf

;~ Make sure chrome is front window
$oIE.setfocus()

if isobj($oIE) Then
    consolewrite("Action 1" & @CRLF)

;~  get the IE toolbar
;~  $oIEToolbar=_UIA_getFirstObjectOfElement($oIE,"controltype:=" & $UIA_ToolBarControlTypeId, $treescope_subtree)
    $oIEToolbar=_UIA_getFirstObjectOfElement($oIE,$cToolbarByName, $treescope_subtree)
    if not isobj($oIEToolbar) Then
        _UIA_DumpThemAll($oIE,$treescope_subtree)
    EndIf

consolewrite("Action 2" & @CRLF)
;~  get the addressbar
;~  $oIEAddressBar=_UIA_getFirstObjectOfElement($oIEToolbar,"class:=Chrome_OmniboxView", $treescope_children) ;worked in chrome 28
;~  $oIEAddressBar=_UIA_getFirstObjectOfElement($oIEToolbar,"controltype:=" & $UIA_EditControlTypeId , $treescope_subtree) ;works in chrome 29
;~  $oIEAddressBar=_UIA_getFirstObjectOfElement($oIEToolbar,"name:=Adres- en zoekbalk"  , $treescope_children) ;works in chrome 29
    $oIEAddressBar=_UIA_getObjectByFindAll($oIEToolbar, $cAddressBarByName  , $treescope_subtree) ;works in chrome 29
    if not isobj($oIEAddressbar) Then
        _UIA_DumpThemAll($oIEToolbar,$treescope_subtree)
    EndIf

;~  $oValueP=_UIA_getpattern($oIEAddressBar,$UIA_ValuePatternId)
;~  sleep(2000)

;~  get the value of the addressbar
;~  $myText=""
;~  $oValueP.CurrentValue($myText)
;~  consolewrite("address: " & $myText & @CRLF)

consolewrite("Action 3" & @CRLF)
;~ Get reference to the tabs
    $oIETabs=_UIA_getFirstObjectOfElement($oIE,"controltype:=" & $UIA_TabControlTypeId, $treescope_subtree)
    if not isobj($oIETabs) Then
        _UIA_DumpThemAll($oIE,$treescope_subtree)
    EndIf

;~ Lets open a new tab within chrome

consolewrite("Action 4" & @CRLF)
;~  $oIENewTab= _UIA_getFirstObjectOfElement($oIETabs,"controltype:=" & $UIA_ButtonControlTypeId, $treescope_subtree)
    $oIENewTab= _UIA_getObjectByFindAll($oIETabs, $cIENewTabByName,$treescope_subtree)
    if not isobj($oIENewTab) Then
        _UIA_DumpThemAll($oIETabs,$treescope_subtree)
    EndIf
;~  _UIA_action($oIENewtab,"leftclick")
;~  sleep(500)

consolewrite("Action 5" & @CRLF)
    $oIEAddressBar=_UIA_getObjectByFindAll($oIEToolbar, $cAddressBarByName  , $treescope_subtree) ;works in chrome 29

    if not isobj($oIEAddressbar) Then
        _UIA_DumpThemAll($oIEToolbar,$treescope_subtree)
    EndIf

    $t=stringsplit(_UIA_getPropertyValue($oIEAddressBar, $UIA_BoundingRectanglePropertyId),";")
    _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    _UIA_action($oIEAddressBar,"leftclick")
    _UIA_action($oIEAddressBar,"setvalue using keys","www.autoitscript.com/{ENTER}")

consolewrite("Action 6" & @CRLF)

;~  give some time to open website
    sleep(2000)
    $oDocument=_UIA_getFirstObjectOfElement($oIE,$cDocumentWindow, $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oIE,$treescope_subtree)
    Else
        $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
        _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    EndIf

    sleep(500)

consolewrite("Action 7 retrieve document after clicking a hyperlink" & @CRLF)
    $oForumLink=_UIA_getObjectByFindAll($oDocument,"name:=Forum", $treescope_subtree)
;~ All document items
    if not isobj($oForumLink) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oForumLink,"leftclick")
    sleep(3000)

consolewrite("Action 8 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oIE, $cDocumentWindow , $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oIE,$treescope_subtree)
    Else
        $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
        _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    EndIf

;~ Now we get the searchfield

    $oEdtSearchForum=_UIA_getObjectByFindAll($oDocument,$cEdtSearch, $treescope_subtree)
    if not isobj($oEdtSearchForum) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    _UIA_action($oEdtSearchForum,"focus")
    _UIA_action($oEdtSearchForum,"setvalue using keys","Chrome can easy be automated with ui automation") ; {ENTER}")
    sleep(500)

;~ Exit
;~ Now we press the button, see relative syntax used as the button seems not to have a name its just 2 objects further then search field
    $oBtnSearch=_UIA_getObjectByFindAll($oDocument,$cSearchButton, $treescope_subtree)
    if not isobj($oBtnSearch) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    $t=stringsplit(_UIA_getPropertyValue($oDocument, $UIA_BoundingRectanglePropertyId),";")
    _DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
    sleep(1000)
    _UIA_action($oBtnSearch,"leftclick")
    sleep(2000)

;~ consolewrite("Action 9 first refresh the document control" & @CRLF)

    $oDocument=_UIA_getFirstObjectOfElement($oIE,$cDocumentWindow  , $treescope_subtree)
    if not isobj($oDocument) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    $oHyperlink=_UIA_getObjectByFindAll($oDocument,$cBoardItem, $treescope_subtree)
    if not isobj($oBtnSearch) Then
        _UIA_DumpThemAll($oDocument,$treescope_subtree)
    EndIf
    sleep(1000)
    _UIA_action($oHyperlink,"leftclick")
    sleep(2000)

EndIf

exit

Edited November 18, 2013 by junkew

oceanwaves · September 20, 2013

Hi, junkew. Thanks for your working. Could you provide some examples for events handle of ui automation? Do I need using ObjEvent or ObjCreateInterface to create a IUIAutomationEventHandler interface then to handle the events?

junkew · September 20, 2013

I did not get this stable to work (only calls once focus changed) maybe something to do with threads interfering, however I have had it working in the past with AIO UDF

'?do=embed' frameborder='0' data-embedContent>>

So far some trial code like below

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"
HotKeySet("{ESC}", "Terminate")

;~ #AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

;===============================================================================
#interface "IUnknown"
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"
; Definition
Global $dtagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
        "AddRef dword();" & _
        "Release dword();"
; List
Global $ltagIUnknown = "QueryInterface;" & _
        "AddRef;" & _
        "Release;"
;===============================================================================
;===============================================================================
#interface "IDispatch"
Global Const $sIID_IDispatch = "{00020400-0000-0000-C000-000000000046}"
; Definition
Global $dtagIDispatch = $dtagIUnknown & _
        "GetTypeInfoCount hresult(dword*);" & _
        "GetTypeInfo hresult(dword;dword;ptr*);" & _
        "GetIDsOfNames hresult(ptr;ptr;dword;dword;ptr);" & _
        "Invoke hresult(dword;ptr;dword;word;ptr;ptr;ptr;ptr);"
; List
Global $ltagIDispatch = $ltagIUnknown & _
        "GetTypeInfoCount;" & _
        "GetTypeInfo;" & _
        "GetIDsOfNames;" & _
        "Invoke;"
;===============================================================================
; #FUNCTION# ====================================================================================================================
; Name...........: _UIA_getPropertyValue($obj, $id)
; Description ...: Get an object from a DTAG
; Syntax.........:
; Parameters ....:
;
; Return values .: Success      - Returns 1
;                  Failure      - Returns 0 and sets @error on errors:
;                  |@error=1     - UI automation failed
;                  |@error=2     - UI automation desktop failed
; Author ........: TRANCEXX
; Modified.......:
; Remarks .......: None
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
;~ http://www.autoitscript.com/forum/topic/153859-objevent-possible-with-addfocuschangedeventhandler/
Func UIA_ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface)
    Local Const $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    $tagInterface = $tagIUnknown & $tagInterface
    Local Const $PTR_SIZE = DllStructGetSize(DllStructCreate("ptr"))
    ; Below line really simple even though it looks super complex. It's just written weird to fit one line, not to steal your eyes
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace($tagInterface, "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "variant*", "ptr"), "hresult", "long"), "bstr", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams
    ; Allocation. Read http://msdn.microsoft.com/en-us/library/ms810466.aspx to see why like this (object + methods):
    $tInterface = DllStructCreate("ptr[" & $iUbound + 1 & "]")
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        DllStructSetData($tInterface, 1, DllCallbackGetPtr(DllCallbackRegister($sMethod, $sRet, $sParams)), $i + 2) ; Freeing is left to AutoIt.
    Next
    DllStructSetData($tInterface, 1, DllStructGetPtr($tInterface) + $PTR_SIZE) ; Interface method pointers are actually pointer size away
    Return ObjCreateInterface(DllStructGetPtr($tInterface), "", $tagInterface, False) ; and first pointer is object pointer that's wrapped
EndFunc


;~ Focus Changed Handler
Global $tFCEHandler
Global $oFCEHandler = UIA_ObjectFromTag("_MyHandler_", $dtagIUIAutomationFocusChangedEventHandler, $tFCEHandler)

$hr=$objUIAutomation.AddFocusChangedEventHandler(0, $oFCEHandler())
$running=true

while $running=true
    consolewrite("running")
    sleep(100)
WEnd
    consolewrite("Exiting2")
$objUIAutomation.RemoveFocusChangedEventHandler($oFCEHandler())
    consolewrite("Exiting3")
$objUIAutomation.RemoveAllEventHandlers()
    consolewrite("Exiting4")
exit

; The End
Func Terminate()
    consolewrite("Exiting")
    $running=false
;~  Exit 0
EndFunc   ;==>Terminate

; Handler methods. "_MyHandler_" is the specified prefix:
Func _MyHandler_QueryInterface($pSelf, $pRIID, $pObj)
    Local $tStruct = DllStructCreate("ptr", $pObj)
    Switch _WinAPI_StringFromGUID($pRIID)
        Case $sIID_IUnknown, $sIID_IUIAutomationFocusChangedEventHandler
        Case Else
            ConsoleWrite(@CRLF)
            Return 0x80004002 ; E_NOINTERFACE
    EndSwitch
    DllStructSetData($tStruct, 1, $pSelf)
    Return 0 ; S_OK
EndFunc
Func _MyHandler_AddRef($pSelf)
    #forceref $pSelf
    Return 1
EndFunc
Func _MyHandler_Release($pSelf)
    #forceref $pSelf
    Return 1
EndFunc
Func _MyHandler_HandleFocusChangedEvent($pSelf, $pElem)
    consolewrite("Focus changing")
    #forceref $pSelf
    If $pElem Then
        Local $oUIElement = ObjCreateInterface($pElem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
;~         Local $sFocused
;~         $oUIElement.CurrentClassName($sFocused)
   $oUIElement.addref()
        ConsoleWrite("Title is: <" &  _UIA_getPropertyValue($oUIElement,$UIA_NamePropertyId) &  ">" & @TAB _
                    & "Class   := <" & _UIA_getPropertyValue($oUIElement,$uia_classnamepropertyid) &  ">" & @TAB _
                    & "controltype:= <" &  _UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId) &  ">" & @TAB  _
                    & " (" &  hex(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId)) &  ")" & @TAB & @CRLF)
    EndIf
;~     Return 0 ; S_OK
EndFunc

Edited November 5, 2013 by junkew

junkew · September 29, 2013

Not sure why but to get the eventhandling work I had to add

$oUIElement.addref()

full function should be

Func _MyHandler_HandleFocusChangedEvent($pSelf, $pElem)
    consolewrite("Focus changing")
    #forceref $pSelf
    If $pElem Then
        Local $oUIElement = ObjCreateInterface($pElem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        $oUIElement.addref()

;~         Local $sFocused
;~         $oUIElement.CurrentClassName($sFocused)

        ConsoleWrite("Title is: <" &  _UIA_getPropertyValue($oUIElement,$UIA_NamePropertyId) &  ">" & @TAB _
                    & "Class   := <" & _UIA_getPropertyValue($oUIElement,$uia_classnamepropertyid) &  ">" & @TAB _
                    & "controltype:= <" &  _UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId) &  ">" & @TAB  _
                    & " (" &  hex(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId)) &  ")" & @TAB & @CRLF)
    EndIf
;~     Return 0 ; S_OK
EndFunc

Edited May 5, 2019 by junkew

Proph · October 6, 2013

This is amazing!! Great job on this!

I was wondering if someone could write an example for me.

Basically I want to automate Ad-Aware Antivirus.

I just want someone to help get me started so I can learn how to use this new set of UDF's.

Here is the first window:

I want to detect the "Scan" link on the top Right and then click it.

Then in the second window:

I want to detect then click on the "Scan Button".

Please show me the most logical way to use these udf's to do these tasks. I think from there I should be able to figure out the rest.

Once again... Great work!!! This looks really cool!! It should come in handy for sure!!

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

junkew

junkew

junkew

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Similar Content