Leaderboard

Automate all windows and browser applications with one UDF function library. Based on the microsoft automation API this library high level supports Recognition of conttrols from EDGE, Chrome, FF, Opera, Safari and Windows native apps Small testing framework to split object repository from coding away Introduction Quickstart - Getting started quickly Simple scripts With this module you can automate all applications/programs that support ui automation and/or accesibility api from microsoft you can recognize more controls than AutoIT can recognize "out of the box" you can use concepts from other testing frameworks like http://download.freedesktop.org/ldtp/doc/ldtp-tutorial.pdf http://safsdev.sourceforge.net/Default.htm coded ui testing from microsoft Some of those controls / applications are chrome browser (partly mainwindow has to be done with MSAA for navigating) chrome://accessibility in the adress bar of chrome or start with "--force-renderer-accessibility" silverlight controls Ribbon control controlbars of Excel/Word IE and FF browsers Windows Media Player Windows clock AFX .. controls (partly) .... Based on the initial AIO Object I now have made the interface file to work with objCreateInterface function which is in the latest beta's automate clicking and querying basic information It gives you a lot of basic information to be able to automate clicking, querying basic information where it goes further in certain situations than AutoIt is identifying Starting threads for background on the ui automation api of microsoft (not for starters) http://en.wikipedia.org/wiki/Microsoft_UI_Automation http://msdn.microsoft.com/en-us/library/ms747327.aspx Previous threads in general help/support Interface AutoItObject IUIAutomation ObjCreateInterface and struct tagPoint in method ElementFromPoint Be aware that API is not allways installed under XP/Vista see http://support.microsoft.com/kb/971513 Within Windows 7 and Windows 8 it should be preinstalled by default. Be aware on 32 and 64 bits way of running your script #AutoIt3Wrapper_UseX64=Y or N Basic example of usage / showing and retrieving the default information, will post multiple examples later Hover your mouse to an area of interest and press ctrl+w and information will be shown in the edit box of the form Simple spy demo (see simplespy.au3 or use latest ZIP attachment for latest version) Main features Recognize windows and html controls for the major browsers Logical and physical description for controls (UI mapping, Application map) Simple repository logic to abstract logical and physical descriptions Store Runtime Type Information in RTI. variables Rubberbanding/highlighting of objects Simple spy to help in making / identifying the physical description Support of regular expression(s) in identifying objects recognize objects on multiple properties supported properties: name ,title, automationid, classname, class, iaccessiblevalue, iaccessiblechildId, controltype, processid, acceleratorkey The actions provided so far "leftclick", "left", "click", "leftdoubleclick", "leftdouble", "doubleclick", _ "rightclick", "right", "rightdoubleclick", "rightdouble", _ "middleclick", "middle", "middledoubleclick", "middledouble", "mousemove", "movemouse" "setvalue","settextvalue" "setvalue using keys" "setValue using clipboard" "getvalue" "sendkeys", "enterstring", "type", "typetext" "invoke" "focus", "setfocus", "activate" "close" "move","setposition" "resize" "minimize", "maximize", "normal", "close", "exist", "exists" "searchcontext", "context" "highlight" "getobject","object" "attach" "capture","screenshot", "takescreenshot" "dump", "dumpthemall" "propertyvalue", "property" match on multiple properties like: name:=((Zoeken.*)|(Find.*)); ControlType:=Button; acceleratorkey:=Ctrl+F Support for 117 different properties see $UIA_propertiesSupportedArray in uiawrappers like for example title, regexptitle, class, regexpclass, iaccessiblevalue, iaccessiblechildid, name, accesskey, automationid, classname IAccessible, IAccessible2, ISimpleDom interfaces debuglogging to a file log.txt (no output in scitewindow) Examples Example 1 Iterating thru the different ways of representing the objects in the tree (#comment-1105548) Example 2 Finding the taskbar and clicking on the start menu button (#comment-1105680) Example 3 Clicking a litlle more and in the end displaying all items from the clock (thats not directly possible with AU3Info) (#comment-1108849) Example 4 that demonstrates the calculator Example 5 Automating chrome Example 6 Demonstrates all stuff within chrome to navigate html pages, find hyperlink, click hyperlink, find picture, click picture, enter data in inputbox Example 7 The chrome example modified to a firefox example Example 8 The other major browser Internet Explorer automated (made on Example 6 and 7) Example 9 Windows media player Example 10 Automating mach 3 (AFX windows and other hard to get recognized by AutoIT) Lot of links are broken due to forum upgrade just search for the text like "Example 11 Demonstrate Word, Notepad and Calculator actions" Example 11 Demonstrate Word, Notepad and Calculator actions ... Example 13 Details 1 about the right pane of the windows explorer Example 14 Details 2 about the right pane of the windows explorer Example 15 Details 3 about the right pane of the windows explorer Example 16 Details 4 about the right pane of the windows explorer Example 17 Details 5 about the right pane of the windows explorer WITH CACHING Example 18 Details 6 about the right pane of the windows explorer WITH VIRTUAL ITEMS Example 19 Eventhandling examples Example 20 Eventhandling examples Example 21a Eventhandling examples Internet Explorer Example 21b Eventhandling examples Internet Explorer Example 22 Eventhandling examples Follow focus Example 23 Eventhandling examples structure changed Example 24 Eventhandling examples IUIAutomationEventHandler Example 25 SAFEARRAYS Example 26 IACCESSIBLE / MSAA Example 27 IACCESSIBLE2 / MSAA Example 28 IACCESSIBLE / MSAA events Example 29 IACCESSIBLE2 events Example 30 ISimpleDOM Example 31 Notepad window move, maximize, minimize Example 32 Three browsers doing the same stuff with small differences in scripting only .. TODO Build recorder Enhance the spy with a nicer UI UI for the repository (now in the script with dot notation) Enhance mapping / identifying on multiple properties instead of 1 combined with index If speed becomes an issue use the caching logic of the MS UIA framework Add the other patterns later Generalize the concept of System Under Test of starting the SUT (for testing framework purposes) Remote running of scripts Fix issue on finding within dynamic context ... edit august 18th 2013 initial post Only zip files are needed to download , just unzip in 1 directory edit july 2016 Made V0_63 and examples works with AutoIt v3.3.14 Windows 10 tested Simple spy gives some basic code as a present Chrome latest versions seems to be having issues with IUIAutomation on tabs/buttons of mainwindow use MSAA for accessing tabsheets / buttons more cleanup to be in UDF style More comments in the source see changelog.txt for previous changes edit september 2017 All examples fixed for the IE, Firefox and Chrome browser Some small but essential fixes in UIAWrappers edit april 2018 Enhanced logic on fallback / dynamic search, still not perfect, to slow Retested with latest Chrome, FF, Edge and IE11 and some extensions to show how to get text from the webpage (examples 5,6,7) Some small bugfixes Some comments as given in forum incorporated edit may 2019 Speed enhancements on especially fallback searching UIA.CFG works now in a better way to turn on/off debug, highlighting, debug2file More stable and consistent behavior Internal cleanup and refactoring of bigger functions Checked with W10 (not tested on W7) Added some W10 properties Run with 3.3.14.5 on W10 UIA_V0_51.zip EXAMPLES_V0_5.zip UIA_V0_63.zip EXAMPLES_V0_63.zip UIA_V0_64.zip EXAMPLES_V0_64.zip EXAMPLES_V0_66.zip UIA_V0_66.zip EXAMPLES_V0_70.zip UIA_V0_70.zip

You need the port at two places: _WD_Option('Port', <PORT>) _WD_Option('DriverParams', --port=<PORT> ...) See my post above. Simple replace ... _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"') by this ... _WD_Option('DriverParams', '--port=9515 --log-path="' & @ScriptDir & '\chrome.log"') ... and it should work. Otherwise there are more issues, but I don't think so 😀 . Best regards Sven

Hello @k_u_x, to be honest, when it comes to speed, Autoit is not the best choice. A while loop without a sleep, like cv.waitKey(10) will put too much pressure on the CPU. Concerning speeding up the search process, you can resize the image to a smaller size before performing the search. _OpenCV_GetDesktopScreenMat is more a proof a concept than a full speed function. It is better to copy the data in an existing data that creating a new one at each loop. You can also cv.UMat everywhere an Input/Ouput Array is expected. If done correctly, it will use your GPU instead of your CPU. It is also not sure that the searching process is slow, you need timings to confirm the slower parts. For inspiration, have a look at https://github.com/smbape/node-autoit-opencv-com/blob/main/samples/dnn/object_detection/object_detection.au3 https://github.com/smbape/node-autoit-opencv-com/blob/main/samples/tutorial_code/tapi/bgfg_segm.au3

Moved to the appropriate AutoIt General Help and Support forum, as the Developer General Discussion forum very clearly states: Moderation Team

...the same way you coded Func Link(). Does chrome save browser data when incognito ?. Maybe you don't have to code anything. Or maybe have a virtual machine to browse whatever you'd like to not have in your PC.

Did you open the helpfile and check?

For $iCount = 1 To 10 .. your code .. Next

I modified one of my old script from 2014: ;Coded by UEZ #include <GUIConstantsEx.au3> #include <GDIPlus.au3> _GDIPlus_Startup() Global Const $STM_SETIMAGE = 0x0172 Global Const $hGUI = GUICreate("GDI+ Test", 200, 100) GUISetBkColor(0x505050) Global Const $iPicBtn = GUICtrlCreatePic("", 50, 28, 100, 44) Global $aButtons = _GDIPlus_BitmapCreateRoundedButtonAndText("install", 100, 44) _WinAPI_DeleteObject(GUICtrlSendMsg($iPicBtn, $STM_SETIMAGE, $IMAGE_BITMAP, $aButtons[0])) GUISetState() Global $aMouseInfo, $bShow = False, $bHide = False Do If WinActive($hGUI) Then $aMouseInfo = GUIGetCursorInfo($hGUI) ;hover simulation Switch $aMouseInfo[4] Case $iPicBtn _WinAPI_DeleteObject(GUICtrlSendMsg($iPicBtn, $STM_SETIMAGE, $IMAGE_BITMAP, $aButtons[1])) $bShow = True $bHide = False Case Else _WinAPI_DeleteObject(GUICtrlSendMsg($iPicBtn, $STM_SETIMAGE, $IMAGE_BITMAP, $aButtons[0])) $bHide = True $bShow = False EndSwitch EndIf Switch GUIGetMsg() Case $GUI_EVENT_CLOSE _WinAPI_DeleteObject($aButtons[0]) _WinAPI_DeleteObject($aButtons[1]) _GDIPlus_Shutdown() Exit Case $iPicBtn MsgBox(0, "Information", "Button pressed") EndSwitch Until False ; #FUNCTION# ==================================================================================================================== ; Name ..........: _GDIPlus_BitmapCreateRoundedButtonAndText ; Description ...: Draw rounded button ; Syntax ........: _GDIPlus_BitmapCreateRoundedButtonAndText($sString, $iWidth, $iHeight[, $iBgColor = 0xFF1BA0E1[, $iFontSize = 16[, $sFont = "Times New Roman"[, ; $iHoverColor = 0xFFC9388C[, $iFrameSize = 2[, $iFontFrameColor = 0x408AD5EA[, $iFontColor = 0xFFFFFFFF]]]]]]) ; Parameters ....: $sString - A string value. ; $iWidth - An integer value. ; $iHeight - An integer value. ; $iBgColor - [optional] An integer value. Default is 0xFF1BA0E1. ; $iFontSize - [optional] An integer value. Default is 16. ; $sFont - [optional] A string value. Default is "Times New Roman". ; $iHoverColor - [optional] An integer value. Default is 0xFFC9388C. ; $iFrameSize - [optional] An integer value. Default is 2. ; $iFontFrameColor - [optional] An integer value. Default is 0x408AD5EA. ; $iFontColor - [optional] An integer value. Default is 0xFFFFFFFF. ; Return values .: an array with 2 GDI bitmap handles -> [0]: default button, [1]: hover button ; Author ........: UEZ ; Version .......: 0.85 build 2025-01-12 ; Modified ......: ; Remarks .......: Dispose returned GDI bitmap handles when done ; Example .......: Yes ; =============================================================================================================================== Func _GDIPlus_BitmapCreateRoundedButtonAndText($sString, $iWidth, $iHeight, $iBgColor = 0xFF1BA0E1, $iFontSize = 16, $sFont = "Times New Roman", $iHoverColor = 0xF0FFFFFF, $iFrameSize = 2, $iFontFrameColor = 0x408AD5EA, $iFontColor = 0xFFFFFFFF) ;some checks If $sString = "" Then Return SetError(1, 0, 0) If Int($iWidth) < $iFrameSize * 2 Then Return SetError(2, 0, 0) If Int($iHeight) < $iFrameSize * 2 Then Return SetError(3, 0, 0) ;create font objects Local Const $hFormat = _GDIPlus_StringFormatCreate() Local Const $hFamily = _GDIPlus_FontFamilyCreate($sFont) Local $tLayout = _GDIPlus_RectFCreate(0, 0, $iWidth, $iHeight) _GDIPlus_StringFormatSetAlign($hFormat, 1) ;center string on X axis _GDIPlus_StringFormatSetLineAlign($hFormat, 1) ;center string on Y axis ;create bitmap and graphics context handles Local Const $aBitmaps[2] = [_GDIPlus_BitmapCreateFromScan0($iWidth, $iHeight), _GDIPlus_BitmapCreateFromScan0($iWidth, $iHeight)] Local Const $aGfxCtxt[2] = [_GDIPlus_ImageGetGraphicsContext($aBitmaps[0]), _GDIPlus_ImageGetGraphicsContext($aBitmaps[1])] ;set drawing quality _GDIPlus_GraphicsSetSmoothingMode($aGfxCtxt[0], $GDIP_SMOOTHINGMODE_HIGHQUALITY) _GDIPlus_GraphicsSetSmoothingMode($aGfxCtxt[1], $GDIP_SMOOTHINGMODE_HIGHQUALITY) _GDIPlus_GraphicsSetTextRenderingHint($aGfxCtxt[0], $GDIP_TEXTRENDERINGHINTANTIALIASGRIDFIT) ;define brush and pen objects Local Const $hBrushFontColor = _GDIPlus_BrushCreateSolid($iFontColor) ;, $hBrushBGColor = _GDIPlus_BrushCreateSolid($iBgColor) Local Const $hPenFontFrameColor = _GDIPlus_PenCreate($iFontFrameColor, $iFrameSize), $hPenHoverColor = _GDIPlus_PenCreate($iHoverColor, $iFrameSize) ;create path object Local Const $hPath = _GDIPlus_PathCreate() ;create cloned path object for string measurement Local Const $hPath_Dummy = _GDIPlus_PathClone($hPath) _GDIPlus_PathAddString($hPath_Dummy, $sString, $tLayout, $hFamily, 0, $iFontSize, $hFormat) _GDIPlus_PathStartFigure($hPath) Local $fArcSize = $iWidth * 0.33333 _GDIPlus_PathAddArc($hPath, $iFrameSize, $iHeight - $fArcSize - $iFrameSize, $fArcSize, $fArcSize, 180, -90) ;BR _GDIPlus_PathAddArc($hPath, $iWidth - $fArcSize - $iFrameSize, $iHeight - $fArcSize - $iFrameSize, $fArcSize, $fArcSize, -270, -90) ;BL _GDIPlus_PathAddArc($hPath, $iWidth - $fArcSize - $iFrameSize, $iFrameSize, $fArcSize, $fArcSize, 0, -90) ;TR _GDIPlus_PathAddArc($hPath, $iFrameSize, $iFrameSize, $fArcSize, $fArcSize, -90, -90) ;TL _GDIPlus_PathCloseFigure($hPath) Local Const $hPath_Clone = _GDIPlus_PathClone($hPath) Local Const $hBrushBGColor = _GDIPlus_PathBrushCreateFromPath($hPath) _GDIPlus_PathBrushSetSurroundColor($hBrushBGColor, $iBgColor) _GDIPlus_PathBrushSetCenterColor($hBrushBGColor, 0xFFFFFFFF) _GDIPlus_PathBrushSetCenterPoint($hBrushBGColor, $iWidth / 2, $iHeight / 2) _GDIPlus_PathBrushSetSigmaBlend($hBrushBGColor, 1, 0.33333) _GDIPlus_GraphicsFillPath($aGfxCtxt[0], $hPath, $hBrushBGColor) _GDIPlus_GraphicsDrawPath($aGfxCtxt[0], $hPath, $hPenFontFrameColor) _GDIPlus_PathReset($hPath) ;add string to path _GDIPlus_PathAddString($hPath, $sString, $tLayout, $hFamily, 1, $iFontSize, $hFormat) ;clear bitmap and draw string _GDIPlus_GraphicsFillPath($aGfxCtxt[0], $hPath, $hBrushFontColor) _GDIPlus_GraphicsDrawPath($aGfxCtxt[0], $hPath, $hPenFontFrameColor) ;draw rectangle on cloned bitmap for hover effect _GDIPlus_GraphicsDrawImageRect($aGfxCtxt[1], $aBitmaps[0], 0, 0, $iWidth, $iHeight) _GDIPlus_GraphicsDrawPath($aGfxCtxt[1], $hPath_Clone, $hPenHoverColor) ;dispose object resources _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_StringFormatDispose($hFormat) _GDIPlus_PathDispose($hPath) _GDIPlus_PathDispose($hPath_Dummy) _GDIPlus_PathDispose($hPath_Clone) _GDIPlus_GraphicsDispose($aGfxCtxt[0]) _GDIPlus_GraphicsDispose($aGfxCtxt[1]) _GDIPlus_BrushDispose($hBrushFontColor) _GDIPlus_BrushDispose($hBrushBGColor) _GDIPlus_PenDispose($hPenFontFrameColor) _GDIPlus_PenDispose($hPenHoverColor) ;create GDI bitmap for later usage Local $aHBitmaps[2] = [_GDIPlus_BitmapCreateHBITMAPFromBitmap($aBitmaps[0]), _GDIPlus_BitmapCreateHBITMAPFromBitmap($aBitmaps[1])] ;dispose GDI+ bitmaps _GDIPlus_BitmapDispose($aBitmaps[0]) _GDIPlus_BitmapDispose($aBitmaps[1]) Return $aHBitmaps EndFunc ;==>_GDIPlus_BitmapCreateRoundedButtonAndText

Thanks bwochinski, it looks like we were posting at the same time. What I really need is a help file that will read my mind!