WebDriver UDF - Help & Support (III)

[seadoggie01]

@ReM You'll want _WD_ElementAction with either attribute or property as the command and id as the option... I don't remember which, but you can figure that out

[Danp2]

@ReM @seadoggie01 has pointed you in the right direction. Curious about why you need to retrieve the ID at runtime. Can't you just obtain it using the browser's developer tools?

P.S. Remember that not all elements will have an associated ID. 😉

[RDE]

On 5/11/2021 at 1:08 AM, Danp2 said:

@RDE I suspect that you may be encountering a rights issue. Try moving the script, the chromedriver executable and your log file to the another directory, preferable not beneath C:\Program Files (x86).

I forgot to say earlier that I tried with Firefox and Edge.

With both Firefox AND Edge, I got the Google page, then the Yahoo page which was promptly replaced by a second copy of the Google page. That's so bizarre!

MOD: Please Use this support topic for support questions in stead of the Examples Topic! Merged.  

Edited May 12, 2021 by Jos

[ReM]

I tried 

WD_ElementAction (Session,Element, “attribute”,“id” ) and it’s function.

Great!!!

I thought the option is only for setting an attribute value.

@seadoggie01 @Danp2 Thank you 

 

Edited May 13, 2021 by ReM

[Danp2]

@RDE It sounds like you are making some progress. Not sure about the subsequent behavior that you encountered. Are you only running the DemoNavigation routine? It does perform some of the actions that you described. Check out the function in wd_demo to get a better understanding of what's occurring.

[Danp2]

@ReM This thread is intended for UDF development discussion only. Please move this conversation to the GH&S thread (see my sig).

[RDE]

1 hour ago, Danp2 said:

@RDE It sounds like you are making some progress. Not sure about the subsequent behavior that you encountered. Are you only running the DemoNavigation routine? It does perform some of the actions that you described. Check out the function in wd_demo to get a better understanding of what's occurring.

You are right, thanks to your help I am making progress. I suspect part of my problem was caused by the mis-match between what google and yahoo show in USA and UK. I've attached screen grabs of what I see – both of them require a response before anything else happens.

Anyway, onwards and upwards, proper understanding will come. Many thanks for your help.

[seadoggie01]

@RDE I'd imagine that once you accept these, they will create a cookie. You'll be able to download the cookie and reupload it on subsequent sessions (see _WD_Cookies). Unfortunately, I'm not in the UK, so I can't check this  I've used this to get websites to recognize my device and skip 2 factor authentication each time

[piepen]

Hi,

Firstly thank you for your work on bringing the Chrome Driver to AutoIT. I am finding that it works very well  

I have been putting together a script that allows cyberark to automate login to some websites. An end user will log in to CyberArk and request a connection. CyberArk then launches an RDP session to what they call their PSM server, it is an RDS Host using RemoteApp capability. The compiled au3 file is launched as a RemoteApp and connection and authentication takes places.

CyberArk provide a WebForm capability but it is limited and too restrictive for my specific requirements.

My initial script script works a treat and automates the login process to a number of different websites. However I found an issue. I had hardcoded the port as 9515 like this:

_WD_Oprtion('Port', 9515)

Which causes a problem if more than one Web Driver session is running at a time. The 2nd session actually launches within the RDP session of the 1st user to connect. Kind of make sense if both users RDP sessions are calling Chromedriver.exe on the same port number. It is a small window of risk but a significant one.

So I added a function to check for in use ports and only use one that is not currently in use. Which is where I found my problem. I cannot get the chrome driver to use anything else other than 9515 now. Even if I hardcode the value like this:

_WD_Option('Port', 9517)

It still connects on 9515 😕 

When I get the value of

_WD_Option('Port')

it returns a value of 9517. Yet the ChromeDriver log clearly shows it connecting on 9515.

I have tried logging off and on again, deleting the Chrome UserData folder for my user. I have search through this support site, SO, and any other site I find using a Google search

So,

1) Why can't I change the port? What am I doing wrong? Ideas, help, suggestions all welcome

2) Is there a better way of doing this, using the ChromeDriverService for example? 

This is the function I am configuring _WD_Options with:

Func _SetupChrome(Const $sWdPath, Const $iPort, $sLogLevel = Default)

   LogWrite("_SetupChrome: Setting Chrome Web Driver Path to " & $sWdPath)
   _WD_Option('Driver', $sWdPath) ; set the path to the web driver
   LogWrite("_SetupChrome: Setting Chrome Web Driver Port to " & $iPort)
   _WD_Option('Port', $iPort) ; set the port the web driver will use

   If ($sLogLevel = "Verbose") Then
      LogWrite("_SetupChrome: WARNING Setting Chrome Web Driver logs to VERBOSE, path = " & $sWdLogPath)
      LogWrite("_SetupChrome: VERBOSE LOGGING CAN RESULT IN PASSWORDS BEING RECORDED IN THE LOG FILE.")
      _WD_Option('DriverParams', '--verbose --log-path="' & $sWdLogPath & '"')
   ElseIf ($sLogLevel = "Warning") Then
      LogWrite("_SetupChrome: Setting Chrome Web Driver to warning, path = " & $sWdLogPath)
      LogWrite("_SetupChrome: WARNING LEVEL LOGGING CAN RESULT IN PASSWORDS BEING RECORDED IN THE LOG FILE.")
      _WD_Option('DriverParams', ' --log-path="' & $sWdLogPath & '"')
   ElseIf ($sLogLevel = Default) Then
      LogWrite("_SetupChrome: Chrome Web Driver logs disabled")
   Else
      LogWrite("_SetupChrome: Unexpected value provided for Log level. Chrome Web Driver logs disabled")
   EndIf

   $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches":' _
    & ' [ "enable-automation"], "useAutomationExtension": false, "args": ["--proxy-pac-url=' & $sChromeAppProxyPacPath & '"] }}}}'
EndFunc

This is how I call the function:

_SetupChrome($sWdExePath, 9517, "Warning")

I will replace the hardcoded 9517 with a variable.

I have opened port 9515 so that the ChromeDriver cannot use it. This is the log message I get:

Quote

[1620987112.021][INFO]: Starting ChromeDriver 90.0.4430.24 (4c6d850f087da467d926e8eddb76550aed655991-refs/branch-heads/4430@{#429}) on port 9515

I am using Chrome Enterprise v 90.0.4430.85

[Danp2]

56 minutes ago, piepen said:

_WD_Option('Port', 9517)

This only tells the UDF which port to use when communicating with the webdriver. You need to also tell the webdriver to use this alternate port number.

See here for prior discussion on this topic.

 

[piepen]

17 minutes ago, Danp2 said:

This only tells the UDF which port to use when communicating with the webdriver. You need to also tell the webdriver to use this alternate port number.

See here for prior discussion on this topic.

 

Thank you  

OK, makes sense. Now I see there is a validation of values that you can pass to DriverParams in the WD_core.au3.

Thanks again

[RDE]

On 5/12/2021 at 12:31 PM, Danp2 said:

@RDE It sounds like you are making some progress. Not sure about the subsequent behavior that you encountered. Are you only running the DemoNavigation routine? It does perform some of the actions that you described. Check out the function in wd_demo to get a better understanding of what's occurring.

My favourite internet site (autosport.com) no longer supports Internet Explorer so my carefully crafted AutoIt scripts no longer work. Thanks to your help, I have made quite a lot of progress in re-writing the scripts using WebDriver UDF. The only real problem I've got now is working out how to replace the function _IEImgGetCollection from IE.au3. This returns a collection object variable representing all the IMG tags in the document.

It feels as if the function _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, $sSelector) should be the answer but I can't get my head around the $_WD_LOCATOR construct nor how $sSelector works. I've seen  $sSelector = "//img[@class='XkWAb-LmsqOc']" or similar in some of the questions/answers but totally failed to understand how it works.

Any pointers you can give me would be very gratefully received.

[Danp2]

@RDE I would suggest using _WD_FindElement, which can return an array of matching element IDs. You can read about the available locator strategies at this link.

How do you decide which images are of interest? For instance, this page contains 24 images. What criteria do you use to select them?

Also, what is your end goal? Will you be downloading the images or something else?

[mLipok]

I have problem with this site:
https://www.imsig.pl/szukaj

I was able to select option. Now I try to to get slected options:

#include "wd_helper.au3"
Global $_IMSIG_WD_SESSION

_Example()
Func _Example()
    _IMSIG__WD_SetupChrome()
    _WD_Navigate($_IMSIG_WD_SESSION, "https://www.imsig.pl/szukaj")
    _WD_LoadWait($_IMSIG_WD_SESSION, 500, 1000 * 10)

     Local $iD = _WD_FindElement($_IMSIG_WD_SESSION, $_WD_LOCATOR_ByXPath, '//select[@name="sekcje[]"]')
     ConsoleWrite("! 2" & @CRLF)
     Local $a_options = _WD_ElementSelectAction($_IMSIG_WD_SESSION, $iD , 'options')
     ConsoleWrite("! 3" & @CRLF)
     _ArrayDisplay($a_options,'$a_options')

EndFunc

Func _IMSIG__WD_SetupChrome()
;~  $_WD_DEBUG = $_WD_DEBUG_Info
    $_WD_DEBUG = $_WD_DEBUG_Error
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    DirCreate(@ScriptDir & '\Log\')
    _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\Log\' & @YEAR & @MON & @MDAY & '-' & @HOUR & @MIN & @SEC & ' Chrome - ZUS.log"')
;~  Local $s_Desired_Capabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" & '] }}}}'
    Local $s_Desired_Capabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}'

    _WD_Startup()

    $_IMSIG_WD_SESSION = _WD_CreateSession($s_Desired_Capabilities)
    Return SetError(@error, @extended, $_IMSIG_WD_SESSION)
EndFunc   ;==>_IMSIG__WD_SetupChrome

I get this error:

Quote

_WD_FindElement ==> Invalid Expression: HTTP status = 200
_WD_ElementSelectAction ==> Invalid Expression

My target functionality is to check which options are currently selected.

Could somebody point me what I do wrong ?

 

Edited May 18, 2021 by mLipok

[Danp2]

35 minutes ago, mLipok said:

_WD_ElementSelectAction ==> Invalid Expression

My guess would be that the square brackets in the element name are causing this error to occur. You could try a few things to fix this --

• Escape the problematic characters
• Use an alternate xpath selector that doesn't include the brackets
• Switch to using a CSS selector
• Etc

I haven't tried it, but maybe something like this will work for you --

Local $iD = _WD_FindElement($_IMSIG_WD_SESSION, $_WD_LOCATOR_ByXPath, '//select[starts-with(@name, "sekcje")]')

 

[mLipok]

23 hours ago, Danp2 said:

I haven't tried it, but maybe something like this will work for you --

Local $iD = _WD_FindElement($_IMSIG_WD_SESSION, $_WD_LOCATOR_ByXPath, '//select[starts-with(@name, "sekcje")]')

 

Checked .. not works.

Also checked with:

Local $iD = _WD_FindElement($_IMSIG_WD_SESSION, $_WD_LOCATOR_ByXPath, '//select[1]')

and I still get:

Quote

_WD_FindElement ==> Invalid Expression: HTTP status = 200
_WD_ElementSelectAction ==> Invalid Expression

In next few hours I will try: $_WD_LOCATOR_ByCSSSelector ... and other possibilities.

[Danp2]

@mLipok The issue appears to be coming from _WD_ElementSelectAction. It helps if you switch to full debugging output --

__WD_Post: URL=HTTP://127.0.0.1:9515/session/bc4656ef1a0f5389deacee71b82b2d06/element; $sData={"using":"xpath","value":"//select[@name=\"sekcje[]\"]"}
__WD_Post: StatusCode=200; ResponseText={"value":{"element-6066-11e4-a52e-4f735466cecf":"ce6eae3e-1eb3-40c5-9368-5bdfdab5ca00"}}...
_WD_FindElement: {"value":{"element-6066-11e4-a52e-4f735466cecf":"ce6eae3e-1eb3-40c5-9368-5bdfdab5ca00"}}
! 2
__WD_Get: URL=HTTP://127.0.0.1:9515/session/bc4656ef1a0f5389deacee71b82b2d06/element/ce6eae3e-1eb3-40c5-9368-5bdfdab5ca00/property/nodeName
__WD_Get: StatusCode=200; $iResult = 0; $sResponseText={"value":"SELECT"}...
_WD_ElementAction: {"value":"SELECT"}...
_WD_FindElement: Selector must be relative when supplying a starting element
_WD_FindElement ==> Invalid Expression: HTTP status = 200
_WD_ElementSelectAction: 
_WD_ElementSelectAction ==> Invalid Expression
! 3

Edit: Just pushed an update to wd_helper.au3 on Github. Please retest with this latest version.

Edited May 19, 2021 by Danp2

[seadoggie01]

Resolving my earlier comment: https://www.autoitscript.com/forum/topic/205553-webdriver-udf-help-support-iii/?do=findComment&comment=1480351

Edge is discarding my tab when it is minimized, causing the session to be invalid. I confirmed this by opening edge://discards/ and toggling the check mark in the "Auto Discardable" box of the tab. Then I minimized the window and let it go for half an hour without touching it. I'll likely edit soon this with a function that solves this by checking the title of a tab and toggling the checkbox.

 

[Danp2]

@seadoggie01 I'd like to take a closer look at this. Can you provide a simple replication script?

[seadoggie01]

Here's the simplest reproducer I have... I haven't automated the discarding yet as there are some shadow root elements on the page, so you'll need to follow the comment's instructions

#include <wd_core.au3>
_WD_Option('Driver', @ScriptDir & "\msedgedriver.exe")
_WD_Option('DriverClose', False)
_WD_Option('DriverDetect', True)
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--log-path=' & @ScriptDir & '\webdriver.log')
_WD_Startup()
Local $sSession = _WD_CreateSession('{"capabilities":{"alwaysMatch": {"ms:edgeOptions": {"w3c": true, "prefs": {"plugins.always_open_pdf_externally": true}}}}}')
_WD_Navigate($sSession, "https://www.google.com")
; To discard a tab, open a new tab, navigate to edge://discards/ and click "Urgent Discard" in the Actions column of the tab table
MsgBox(0, "Discard Time", "Please discard the tab, then click OK")
_WD_Navigate($sSession, "https://www.google.com")
MsgBox(0, "WebDriver: Navigate Error", "Error: " & @error & " Extended: " & @extended)
_WD_Shutdown()

Edit: The (un?)expected result is Error 10, extended 404 on the second MsgBox

Edited May 26, 2021 by seadoggie01