IUIAutomation MS framework automate chrome, FF, IE, ....

DrXenos · April 4, 2015

I am on a different version (not published yet) with so many changes that a diff for me is to hard.

What did you change and what was your problem with the version you used.

OK, sorry, I'll try to explain then. None of these "special" properties were working. Setting them to anything caused the search to fail.

Digging into the code, I made the following changes which seems to fix it (I am using the files from "UIA_V0_51.zip" at the start of this thread):

1. in "_UIA_normalizeExpression" at line #853 (just before the "endif" in the "else" block for "if $index >=0"), I added the line:

"$bSkip = true"

2. in "_UIA_getObjectByFindAll" at the start of the loop "For $j = 1 To $arrSize" I added two lines:

$propName = $properties2Match[$j][0]

$propValue = $properties2Match[$j][1]

Without these changes, the code never gets into the "if" block that follows, to process the three special properties, and if it did, it uses $propName and $propValue without ever assigning them.

Regards,

DrX

mLipok · April 9, 2015

the script automatically creates files named like this:

20150401-204424355.xml

How to disable the creation of these files?

mLipok

junkew · April 9, 2015

The intention of the xml files is later to show them in a nice format with an html5 page

save this as index.html to load xml dynamically and in future version nicely formatting

<!DOCTYPE html> 
<html> 
<head> 
    <title>reading xml</title> 
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> 
</head>
<body>
    <input type="file" id="files" name="files[]" multiple />
    <output id="list"></output>

    <script>
      function handleFileSelect(evt) {
        var files = evt.target.files; // FileList object

        // Loop through the FileList
        for (var i = 0, f; f = files[i]; i++) {

          var reader = new FileReader();

          // Closure to capture the file information.
          reader.onload = (function(theFile) {
            return function(e) {
              // Print the contents of the file
    var xmlData = reader.result;  
    var parser=new DOMParser(); 
    var xmlDoc=parser.parseFromString(e.target.result,"text/xml");
                
                // creates a <table> element and a <tbody> element
    var tbl     = document.createElement("table");
    //var strOutHtml='<table width="100%" border="0">';
                var tblBody = document.createElement("tbody");
          
    //
    var xmlData = xmlDoc.getElementsByTagName("logline");
    for(var j=0; j < xmlData.length; j++) {
            var row = document.createElement("tr");   
    //    if (xmlData[j].getAttribute("level") == "5") {
    //        row.setAttribute("bgcolor", "red")
    //    }
        if (xmlData[j].getAttribute("level") == "0") {
            row.setAttribute("bgcolor", "lightgreen")
        }
           var cell = document.createElement("td");
            var cellText = document.createTextNode(xmlData[j].textContent);
            cell.appendChild(cellText);
            row.appendChild(cell);
            tblBody.appendChild(row);           
        
    }
    tbl.appendChild(tblBody);
    
    tbl.setAttribute("border", "2");    
    document.body.appendChild(tbl);

              //}
              //document.getElementById('list').insertBefore(tbl, null);
              // appends <table> into <body>
    // sets the border attribute of tbl to 2;
    

  
              //document.getElementById('list').insertBefore(textNode, null);
              
            };
          })(f);

          // Read in the file
          //reader.readAsDataText(f,UTF-8);
          //reader.readAsText(f);
          reader.readAsText(f);
           
        }
      }

      document.getElementById('files').addEventListener('change', handleFileSelect, false);

    </script>
</body>


...

but to answer your question to turn it off

in

1. uiawrappers

turn things true or false

_UIA_setVar("Global.Debug", True)
    _UIA_setVar("Global.Debug.File", True)
    _UIA_setVar("Global.Highlight", True)

or

2. Create a UIA.CFG file in the directory

content could be like below (could be an issue with the variables stored in the datadictionary object not coming thru nicely and then you should use option 1 as given above)

; This is an inifile for UIA wrappers having the configuration defaults
; Debug=true        Turn debugging on of off with true/false value
; Highlight=true    Turn Highlighting rectangle to true / false 
; TODO: AutoStartSUT=true AutoStartSUT is starting all SUT's automatically

[Global]
Debug=false
Debug.File=false
Highlight=false
AutoStartSUT=false

[Multiuser]
CFGPerUser=false

[Folders]
subfolders=false

;System under test settings
; Folder      = Location where exe can be found
; Workingdir  = Location of the working directory
; exe         = Name of the exe to start
; Fullname    = Path & name of exe 
; Windowstate = minimized, maximized, normal

[SUT1]
Folder=%Windowsdir%\system32
Workingdir=%Windowsdir%\system32
exe=calc.exe
Fullname=%Windowsdir%\system32\calc.exe
Parameters=
Processname=calc.exe
Windowstate=normal

[SUT2]
Folder=%Windowsdir%\system32
Workingdir=%Windowsdir%\system32
exe=notepad.exe
Fullname=%Windowsdir%\system32\notepad.exe
Parameters=
Processname=notepad.exe
Windowstate=normal  

[SUT3]
C:\Program Files (x86)
Folder=%programfilesdir%\Microsoft Office\Office14
Workingdir=%programfilesdir%\Microsoft Office\Office14
exe=winword.exe
Fullname=%programfilesdir%\Microsoft Office\Office14\winword.exe
Parameters=
Processname=winword.exe
Windowstate=normal

Edited April 9, 2015 by junkew

mLipok · April 9, 2015

thanks

Did you mean that I must change

_UIA_setVar("Global.Debug", True)
    _UIA_setVar("Global.Debug.File", True)
    _UIA_setVar("Global.Highlight", True)

inside:

Func _UIA_LoadConfiguration()....

if so then It was not enough

I must alo comment

; _UIA_LogFile(@YEAR & @MON & @MDAY & "-" & @HOUR & @MIN & @SEC & @MSEC & ".XML" , true)

junkew · April 10, 2015

Ok will fix it that xml should not be created in later version

mLipok · April 30, 2015

Simple spy give me something like this:

Mouse position is retrieved 278-301
At least we have an element [Later][CCCommandLink]

Having the following values for all properties: 
Title is: <Later>   Class   := <CCCommandLink>  controltype:= <UIA_ButtonControlTypeId> ,<50000>    , (0000C350)    154;274;438;40
*** Parent Information top down ***
3: Title is: <Pulpit>   Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1280;1024
"Title:=Pulpit;controltype:=UIA_PaneControlTypeId;class:=#32769"    
2: Title is: <Przeglądarka - Internet Explorer>    Class   := <Alternate Modal Top Most>   controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    -4;-4;1288;988
"Title:=Przeglądarka - Internet Explorer;controltype:=UIA_PaneControlTypeId;class:=Alternate Modal Top Most"   
1: Title is: <Java Update Needed>   Class   := <#32770> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    99;110;506;253
"Title:=Java Update Needed;controltype:=UIA_WindowControlTypeId;class:=#32770"  
0: Title is: <Java Update Needed>   Class   := <TaskDialog> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    102;132;500;228
"Title:=Java Update Needed;controltype:=UIA_PaneControlTypeId;class:=TaskDialog"    


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Przeglądarka - Internet Explorer;controltype:=UIA_PaneControlTypeId;class:=Alternate Modal Top Most", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Java Update Needed;controltype:=UIA_WindowControlTypeId;class:=#32770", $treescope_children)   
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Java Update Needed;controltype:=UIA_PaneControlTypeId;class:=TaskDialog", $treescope_children) 
_UIA_Action($oP0,"setfocus")
_UIA_setVar("Later.mainwindow","title:=Later;classname:=CCCommandLink")
_UIA_action("Later.mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_title:= <Later>
UIA_text:= <Later>
UIA_regexptitle:= <Later>
UIA_class:= <CCCommandLink>
UIA_regexpclass:= <CCCommandLink>
UIA_iaccessiblechildId:= <0>
UIA_id:= <CommandLink_202>
UIA_handle:= <591628>
UIA_RuntimeId:= <42;591628>
UIA_BoundingRectangle:= <154;274;438;40>
UIA_ProcessId:= <3000>
UIA_ControlType:= <50000>
UIA_LocalizedControlType:= <przycisk>
UIA_Name:= <Later>
UIA_HasKeyboardFocus:= <True>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <CommandLink_202>
UIA_ClassName:= <CCCommandLink>
UIA_HelpText:= <Continue and you will be reminded to update again later.>
UIA_ClickablePoint:= <373;294>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <591628>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <DirectUI>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Later>
UIA_LegacyIAccessibleDescription:= <Continue and you will be reminded to update again later.>
UIA_LegacyIAccessibleRole:= <43>
UIA_LegacyIAccessibleState:= <1048964>
UIA_LegacyIAccessibleDefaultAction:= <Naciśnij>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:2812,hwnd:0x9070C Override(parent link):Nested [pid:3000,hwnd:0x9070C Override(parent link):Unidentified Provider (unmanaged:comctl32.dll)]; Main:Nested [pid:3000,hwnd:0x9070C Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd:Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

How I can click this Java "Later" button ?

mLipok

EDIT:

I play a little more and here is solution:

Local $oP0a=_UIA_getObjectByFindAll($oP1, "title:=Later;classname:=CCCommandLink", $treescope_children)
_UIA_action($oP0a,"setfocus")
_UIA_action($oP0a,"click")

Edited April 30, 2015 by mLipok

egonp · May 3, 2015

Hello,

I tried to read out the title of a control and then to click on an button with the same name. I used Total Commander's Nagscreen because of only three buttons to choose from. With simplespy I can identify the control with the name/title I would need. But I can't find out the title and have it in value to set later to click on the right button.

Simplespy finds the control:

Mouse position is retrieved 959-644
At least we have an element [3][TPanel]

Having the following values for all properties: 
Title is: <3>   Class   := <TPanel> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    950;637;25;22
*** Parent Information top down ***
4: Title is: <Desktop>  Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1920;1080
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"   
3: Title is: <Total Commander>  Class   := <TNASTYNAGSCREEN>    controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    665;356;590;368
"Title:=Total Commander;controltype:=UIA_WindowControlTypeId;class:=TNASTYNAGSCREEN"    
2: Title is: <> Class   := <TNotebook>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    668;388;584;333
"Title:=;controltype:=UIA_PaneControlTypeId;class:=TNotebook"   
1: Title is: <NagPage>  Class   := <TPage>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    668;388;584;333
"Title:=NagPage;controltype:=UIA_PaneControlTypeId;class:=TPage"    
0: Title is: <> Class   := <TPanel> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    684;631;552;78
"Title:=;controltype:=UIA_PaneControlTypeId;class:=TPanel"  


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Total Commander;controltype:=UIA_WindowControlTypeId;class:=TNASTYNAGSCREEN", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TNotebook", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=NagPage;controltype:=UIA_PaneControlTypeId;class:=TPage", $treescope_children) 
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TPanel", $treescope_children)   
_UIA_Action($oP0,"setfocus")
_UIA_setVar("3.mainwindow","title:=3;classname:=TPanel")
_UIA_action("3.mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_title:= <3>
UIA_text:= <3>
UIA_regexptitle:= <3>
UIA_class:= <TPanel>
UIA_regexpclass:= <TPanel>
UIA_iaccessiblechildId:= <0>
UIA_id:= <3279550>
UIA_handle:= <3279550>
UIA_RuntimeId:= <42;3279550>
UIA_BoundingRectangle:= <950;637;25;22>
UIA_ProcessId:= <5188>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <Bereich>
UIA_Name:= <3>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <3279550>
UIA_ClassName:= <TPanel>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <3279550>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <3>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:5256,hwnd:0x320ABE Main:Nested [pid:5188,hwnd:0x320ABE Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

Now I put the samplecode in Autoit. All is going, a bit slow though, but all elements are focussed well. The code:

;~ Versuch, den Totalcommander zu steuern


#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
;#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

;~ Make this language specific
const $cTCClassName="TNASTYNAGSCREEN"
const $cButton1="1"

$strTCExeFolder="C:\Program Files (x86)\TC UP\"

$strTCStartup=""
$strTCExe=$strTCExeFolder & "TOTALCMD64.EXE"

;~ Start Total Commander
if fileexists($strTCExe) Then
    if not processexists("TOTALCMD64.EXE") Then
        run($strTCExe & $strTCStartup,"", @SW_MAXIMIZE )
        ProcessWait("TOTALCMD64.EXE")
        ;~ Just to give some time to start
        sleep(10000)
    endif
Else
    consolewrite("Total Commander nicht gefunden. Bitte manuell starten:" & @CRLF )
    consolewrite($strTCExe & $strTCStartup & @CRLF)
EndIf

;~ Finde das TC Fenster:

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Total Commander;controltype:=UIA_WindowControlTypeId;class:=TNASTYNAGSCREEN", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TNotebook", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=NagPage;controltype:=UIA_PaneControlTypeId;class:=TPage", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TPanel", $treescope_children)
_UIA_Action($oP0,"setfocus")
Local $okrampf=_UIA_getObjectByFindAll($oP0, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TPanel", $treescope_children)
_UIA_Action($okrampf,"setfocus")


        ConsoleWrite("Title is: <" &  _UIA_getPropertyValue($okrampf,$UIA_NamePropertyId) &  ">" & @TAB _
                    & "Class   := <" & _UIA_getPropertyValue($okrampf,$uia_classnamepropertyid) &  ">" & @TAB _
                    & "controltype:= <" &  _UIA_getPropertyValue($okrampf,$UIA_ControlTypePropertyId) &  ">" & @TAB  _
                    & " (" &  hex(_UIA_getPropertyValue($okrampf,$UIA_ControlTypePropertyId)) &  ")" & @TAB & @CRLF)

Exit

The variable $okrampf, I thought, would deliver the needed information, but no object is found. But... the Pane gets highlighted, it is found!

See the console-output:

>"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "G:\Programmierung\Autoit\Rechnersteuerung\Tutorialskripts 01\tc-ctrl02.au3" /UserParams    
+>14:18:37 Starting AutoIt3Wrapper v.14.801.2025.0 SciTE v.3.4.4.0   Keyboard:00000407  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64    Environment(Language:0407)
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\Egon\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\Egon\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.12.0)  from:C:\Program Files (x86)\AutoIt3  input:G:\Programmierung\Autoit\Rechnersteuerung\Tutorialskripts 01\tc-ctrl02.au3
+>14:18:37 AU3Check ended.rc:0
>Running:(3.3.12.0):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "G:\Programmierung\Autoit\Rechnersteuerung\Tutorialskripts 01\tc-ctrl02.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
Title is: <** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **>    Class   := <** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **>   controltype:= <** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT **>     (00000000) 
+>14:18:51 AutoIt3.exe ended.rc:0
+>14:18:51 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 14.36

Perhaps you find some time to put me in the right direction, please. How can I solve this, how can I address the object and read out the title?

Greetings,

Egon

junkew · May 3, 2015

Best is to use inspect.exe to see if you can see the control in its own hierarchy. Alternative is to use uia dump them all function but thats a little harder. If control not identifiable i would use coordinates within the panel.

egonp · May 3, 2015

Thank you for your prompt answer, Junkew. As a beginner I'm not able to use coordinates within the panel. And I don't know, why simplespy could identify the control and find out the name, but then, in my code there isn't that object. I will give it a try.

Greetings,

Egon

muchado · May 5, 2015

Quickstart
Start simplespy.au3
Hover your mouse over the element you want to handle
Press ctrl+w
Copy / paste the given source to a new AutoIT script
If you need help
ask for help in new thread in general support section and post
screenshot that shows your element is highlighted with simplespy
post output of simplespy with all properties
attached log.txt if available in your script directory
use allways contents of latest zip files which could be work in progress (WIP) zip

This looks as though it is really good. However, I have a couple of questions, which are delaying my start:

1. When SimpleSpy calls UIAWrappers, there is an error for the following variable ($STR_REGEXPMATCH). Where should that variable be set and where?

2. Should I extract each zip file into a separate folders? Should these folders be named and located anywhere specific? How will the functions in them be accessible from other scripts?

junkew · May 5, 2015

1. Which version of AutoIt are you using please use latest beta

2. unzip them all in 1 folder then it should work for sure later on you can read in the help files of autoit how you can install search paths for includes in separate folders

mLipok · May 19, 2015

My question related to JAVA Security warning , I mean something like this: "You are using old Java version it is strongly recomended to update [Update / Run this time / Close ]"

simply spy gives me this

;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=https://someURL - Internet Explorer;controltype:=UIA_WindowControlTypeId;class:=IEFrame", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame Notification Bar", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Powiadomienie;controltype:=UIA_ToolBarControlTypeId;class:=DirectUIHWND", $treescope_children)
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Zamknij.mainwindow", "title:=Zamknij;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Zamknij;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

This is a snippet to close this opened window.

QUESTION:

Currently, if the window does not appear the program, and so does each of these features, including click (sometimes in another part of the screen). It happened to me that the script showed StartUp menu.

How to prevent this?
How to check if this window Appear ?

Best regards,

mLipok

EDIT: WinWait... not working as this windows are "related/embended" to IE11, and Au3Info do not see it.

Edited May 19, 2015 by mLipok

junkew · May 23, 2015

unfortunately thats a difficult question to answer as it heavily depends on what is flowing around in your system.

1. Worst case you have to build a loop with a 100 ms timedelay for 10-20 times with the disbenefit when its not coming you lost 2 seconds of speed.

2. With iuiAutomation you can partly tackle this by watching the events flowing around. See examples on how to see those. As it seems your javawindow is more or less an exceptional window its probably best to catch it by an event and handle exceptional windows in those events and not in your core flow. The events can be compared with winwait although its a little harder.

zreo15 · June 10, 2015

i download cef_binary_3.2378.1280.gb6e5310_windows32_client its chromium embedded browser example client app having chromium embedded framwork 3

its a kind of user created browser based on chromium browsers

i was trying to do iuiautomation on it but simplyspy isnt able to access the elements inside html page being displayed though the other controls of the apllication can be accesed

the purpose of this experiment was because cef3 was created to be embbeded in with other application and it could display the html files just like a browser does and i thought maybe we can use it to create GUI for autoit using it but the problem is html isnt being accesed

junkew · June 10, 2015

I do not have CEF3 examples but if you read this

https://www.chromium.org/developers/design-documents/accessibility

then this is probably the trick to turn accessibility on

Windows: Chrome calls NotifyWinEvent with EVENT_SYSTEM_ALERT and the custom object id of 1. If it subsequently receives a WM_GETOBJECT call for that custom object id, it assumes that assistive technology is running.

Check with assistive tools like NVDA or Google accessibility tools what you get out of GEF. If they work its just a matter of sendmessage from AutoIT sending WM_GETOBJECT when it receives EVENT_SYSTEM_ALERT

zreo15 · June 10, 2015

well i tried chrome://accessibility but the application changes the url to chrome://version and the other trick mentioned there wont run this example file has a exe file and no othr file.....

so i downloaded the win64 binary file now i just have to figure out a way to compile them to launch the client app in force render accessibility or even launch atleast first.....

((aaaaaaaaaa just when i need the cef forums the frikin site is down..... now just gotta try trial and error n launch the app window sumhow....))

Edited June 10, 2015 by zreo15

junkew · June 10, 2015

you probably do not have to do that see below. I installed CEF test app on my PC and below works perfectly also with SimpleSpy

Test

1. Start chrome

2. Enter Chrome://accessibility in addressbar and make sure its off

3. Start the attached script

4. Open new tab, you should see a message from AutoIT that event was send and handshake back was send

5. Go back to chrome://accessibility and refresh

6. Accessibility is now on

As CEF is based on chromium it should respond similar in the handshake

Workinstruction for your scenario

1. start this script

2. start your CEF application

3. start simplespy

#include <APISysConstants.au3>
#include <GUIMenu.au3>
#include <WinAPIProc.au3>
#include <WinAPISys.au3>
#include <WindowsConstants.au3>

Local $hEventProc = DllCallbackRegister('_EventProc', 'none', 'ptr;dword;hwnd;long;long;dword;dword')
Global $g_tRECT, $g_iIndex, $g_hMenu = 0

OnAutoItExitRegister('OnAutoItExit')

Local $hEventHook = _WinAPI_SetWinEventHook($EVENT_SYSTEM_ALERT, $EVENT_SYSTEM_ALERT, DllCallbackGetPtr($hEventProc))

While 1
    Sleep(1000)
WEnd

Func OnAutoItExit()
    _WinAPI_UnhookWinEvent($hEventHook)
    DllCallbackFree($hEventProc)
EndFunc   ;==>OnAutoItExit

Func _EventProc($hEventHook, $iEvent, $hWnd, $iObjectID, $iChildID, $iThreadID, $iEventTime)
    #forceref $hEventHook, $iObjectID, $iChildID, $iThreadID, $iEventTime


    Switch $iEvent
        Case $EVENT_SYSTEM_ALERT
            if $iObjectID=1 then
                $result = _SendMessage($hWnd, $WM_GETOBJECT, 0, 1)
                consolewrite("Chrome gave a call so we replied " & $iObjectID)
            EndIf
    EndSwitch
EndFunc   ;==>_EventProc

Edited June 10, 2015 by junkew
tested it with CEF application

zreo15 · June 10, 2015

yup its working now simplespy is able to detect html elements now but as soon as I close the cefclient window and reopen it and then try to use simple spy it dosent work and i have to run ur script agn to make it work....

i guess now i wil try to run that win64 binarys client app and see there must be a way to permanantly turn accessibility on in that

wells till then how to make this WM_GETOBJECT calls....??

junkew · June 11, 2015

its the other way around. CEF applications broadcast "Hello, anyone there" with $EVENT_SYSTEM_ALERT. Then any screenreader/automation can answer "yes, hello I am here" by sending back $WM_GETOBJECT.

This is the easiest way of doing it. Another way is triggering IAccessible2 interface but then you have to do stuff with OBJ_CREATEINTERFACE and/or AIO UDF logic which is more complicated / work

mLipok · June 21, 2015

@junkew

May I ask you to check and repair links in the OP, as after forum upgrading , the links are broken.

Best regards,

mLipok

Edited June 21, 2015 by mLipok

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

junkew

junkew

junkew

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Similar Content