Attach and Click Element using WebDriver UDF

[JugglingCoder]

I've spent days on this searching high and low (google / these forums) and running my code over and over but still can't get/grasp this implementation.... so, hoping somebody will help me out.

Goal

Using the WebDriver UDF,
1.) On click of a button/hotspot of an AutoIt GUI
2.) "Attach" to an EXISTING Window that's already open (not create a new window)
3.) and then Click on an "a" tag (display:block) that wraps a div

Other Info

The "Existing Window" that is already open will be in a 2nd monitor and or projector screen with varying aspect ratio and or pixel dimensions.  Otherwise I would use just basic AutoIt MouseMove / MouseClick functions. I.E. I have no way to know "where" to click and therefore am trying to use the WebDriver UDF

Window Info (Using "AutoIt v3 Window Info)
The "Title" of the window is:  CJ Media Master - Media Selection — Mozilla Firefox

URL
OFFLINE Site (Running on local WAMP installation)
Path (Virtually Hosted) - http://gm_local/app/app_index.php

WebDriver UDF Status

I have download and properly set up the files.  The "wd_demo.au3" runs fine and running some basic code like "_WD_Navigate" I have been able to implement easily.  I like using FireFox as it is my preferred browser but willing to change to Chrome  or Edge if need be.

Issue

(see code below)  I cannot get "_WD_Attach" to work.  From my understanding, "_WD_Attach" is just like "_IEAttach" and it should let me Attach to the specified instance of a Browser window (i.e. the window I have already open) but for the life of me... I can't get it to work > "_WD_Attach ==> No match".

YES, I am using FireFox and the window is open.

Code

Note that the GUI code is not in the code below (to keep it simple). I have no issues creating clicking on the GUI, just the issue with "Attaching" using the WebDriver UDF and clicking using the WebDriver UDF

expandcollapse popup

#include <Constants.au3>
#include "udf\wd_core.au3"
#include "udf\wd_helper.au3"
#include "udf\wd_capabilities.au3"

Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced, -1 to -4=Nocase

Global $sCapabilities, $sSession
Global $game_window_title = "CJ Media Master" ; the Title of the window
Global $website = "http://www.yahoo.com"

; This will open a new window of FireFox. NOT what I want
;ShellExecute("C:\Program Files\Mozilla Firefox\firefox.exe", "--remote-debugging-port=9222")

SetupGecko()
_WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to load")

$sSession = _WD_CreateSession($sCapabilities)
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to create a session")

;_WD_Navigate($sSession, $website) ; opens a webpage > Works
Sleep(1000)
;   _IEAttach > old > depricated
;_WD_Attach($sSession, $game_window_title, "title") ;  Fail
;_WD_Attach($sSession, $game_window_title) ;  Fail
$my_game_window = _WD_Attach($sSession, $game_window_title) ;  Fail > _WD_Attach ==> No match

; After I get a "match", i.e. a good $my_game_window VAR, I should be able to use 
; _WD_GetElementById to "Click" the DIV with _WD_ElementAction($sSession, $will_click_this, 'click')

Func SetupGecko()
    ; Have no idea why or which "Port" to use or if even this _WD_Option is needed but... the uncommented one below seems to work.
    _WD_Option('Driver', 'udf\geckodriver.exe')
    Local $iPort = _WD_GetFreePort(4444, 4500) ; instead of specifing a specific port, this function will find a free port from one of the ports listed
    _WD_Option('Port', $iPort)
    ;_WD_Option('Port', 9515)
    ;_WD_Option('Port', 4444)
    ;If @error Then Return SetError(@error, @extended, 0)
    ;_WD_Option('Port', $iPort)
    ;_WD_Option('DriverParams', '--port=' & $iPort & ' --log trace')    
    
    ; WebDriver Capabilities
    ; See: https://www.autoitscript.com/wiki/WebDriver_Capabilities
    _WD_CapabilitiesStartup()
    _WD_CapabilitiesAdd('alwaysMatch', 'firefox')
    _WD_CapabilitiesAdd('browserName', 'firefox')
        _WD_CapabilitiesAdd('acceptInsecureCerts', True)
    ;_WD_CapabilitiesAdd('debuggerAddress', 9222)
    ;_WD_CapabilitiesDump(@ScriptLineNumber) ; dump current Capabilities setting to console - only for testing in this demo
    $sCapabilities = _WD_CapabilitiesGet()  
    ; Doesn't work
    ;$sCapabilities = '{"capabilities":{"alwaysMatch":{"acceptInsecureCerts":true, "pageLoadStrategy":"none", "moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:\\Users\\Username\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\Profile.Name"]}}}}' 
    Return $sCapabilities
EndFunc   ; SetupGecko

;_WD_DeleteSession($sSession)
;_WD_ShutDown()

 

[SOLVE-SMART]

Hi @JugglingCoder 👋 ,

actually I would really like to help you, but what you're write and what is seen in your code does not quite agree in my opinion. Please explain your purpose a bit more, because "$game_window_title" refers to gaming related automation which is not allowed in this forum to talk or support about it.

Forum Rules

Best regards
Sven

[JugglingCoder]

Hello SOLVE-SMART,  thank you for taking a look.

"refers to gaming related automation"  This automation has nothing to do with a commercial game.  I am trying to build "automation" so that I can click an AutoIt GUI on my tablet which in turn will click a place on a webpage which is shown on a projector screen. Please see attached image, it should explain a lot.  The "game" is a simple self programmed Trivia type game which I use at events, etc.  With old IE and "#include <IE.au3>" it was simple to click but things have changed and I need to learn/utilize the WebDriver UDF to get the click but am having difficulties.

Basically I'm just trying to "attach" to a currently open widow (the trivia window) and then when clicking a button on the AutoIt GUI it clicks a div in the page that is shown on the projector.

[SOLVE-SMART]

Thanks @JugglingCoder for the explanation. I will answer later on this and suggest a way of doing it.

Best regards
Sven

[Danp2]

How was the browser originally launched? You won't be able to connect to it if it is the standard browser instance without the remote debugging enabled.

[JugglingCoder]

"How was the browser originally launched?"

Quick Answer:  Browser originally launched manually (i.e. by opening with mouse)

Long Answer: The app (trivia game) I built in PHP/javaScript and I launch and play it on a local network (WAMP). When using a tablet to control the game it's impossible to move the mouse "off screen" to the extended monitor (projector screen) with my finger so  since _IEAttach is depreciated  I figured I try an implement the WebDriver UDF method in order to be able to click things over on the projector screen.

Tried and tried, searched high and low because "_WD_Attach" wasn't 'attaching' so... posted here.  yesterday I found this thread "How to attach to an already running Chrome instance?" and Danp2 mentioned, "while you can do this with Firefox, this isn't possible with Chrome." and because I'm using FireFox I went through every page of the WebDriver Help thread but couldn't find anything that solved my issue.

Danp2, just to be clear for my understanding, are you saying that the WebDriver UDF does not work with "manually" opened browser windows?  In other words It's a utility that creates it's own instances and then is able to manipulate only those 'types' of browser windows?

Sure would be nice to 'click' specific elements in a document with an html "id" tag... would have made life easy but if it doesn't work, it doesn't work.

[Danp2]

39 minutes ago, JugglingCoder said:

are you saying that the WebDriver UDF does not work with "manually" opened browser windows?

Yes, unless you launch the browser executable with the correct parameters so that it is accepting the webdriver commands.

[JugglingCoder]

I see, thank you, that clears things up.  I suppose I could initially launch the app with the webdriver and then manipulate it using WebDriver UDF.   Also assuming the webdriver instance is just a normal window I can normally manipulate with normal mouse/finger actions and standard AutoIt functions.  I think that might work... I'll give it a shot.

[JugglingCoder]

Final update for this issue.

Success!  The key was to use an instance of a browser window (in my case FireFox) that was launched by the geckodriver.exe and NOT try to "attach" to an already running instance.  After running the FireFox instance I simply used the "_WD_GetElementById" and "_WD_ElementAction" of the WebDriver UDF to click the appropriate divs (wrapped in a tags).