WebDriver UDF - Help & Support (III)

[RDE]

A program that I've been using on a Monday to Friday basis for some time failed to load its web pages today so it threw up a load of errors that all had their roots in the fact that the expected page wasn't present. It was OK on Monday and Tuesday and hasn't been changed in any way for weeks!

Just in case, I downloaded the latest release and got the same (lack of) result. I brainstormed myself into running wd_demo.au3 and got the same lack of web pages.

I've attached the console outputs from running wd_demo.au3 on 3 different browsers. There are no log files in the stated log-path.

According to Windows update history, the most recent Quality update was "Successfully installed on ‎03/‎09/‎2021" and Feature update to Windows 10, version 2004 was "Successfully installed on ‎03/‎08/‎2020".  

Any clues please?

 

ConsoleReport.txt

[Danp2]

It's like it can't find your Webdriver executable. Do they reside in your working directory (D:\Data\_WebDriver\)? If that's isn't the cause, then I would guess some form of rights issue.

Edit: Another thing to check is your AV software, which could be detecting AutoIt and preventing it from launching the other executable.

Edited September 9, 2021 by Danp2

[EagerLearner]

Just came across an old thread by chance while trying to learn how to use WD, a user mentioned the new tab function wouldn't work after closing a tab and @Danp2 replied with new code that worked. As far as I can tell, this updated code was never integrated into the current release of WD on Git. Wanted to suggest the update. If I'm wrong or doing something wrong, please let me know as I am still very very new to programming in general.
 

On a sidenote, I'm just having a ton of trouble getting started since there is a lack of documentation and my noobness in programming. With WD do I have to use Gecko/Chrome with the start session parameters or is it possible to simply take control of a current/active window? My current goal is simply to create a new tab in my current user controlled session, followed by clicking on some buttons on the page. I've spent two days and have so far managed to create a new session and a new tab...but am wondering if I'm even on the correct path.

[Danp2]

@EagerLearnerThanks for the report on the new tab issue. For your other questions, please take a look at the Webdriver Wiki entry. There's a FAQ section that addresses some of this.

[RDE]

14 hours ago, Danp2 said:

It's like it can't find your Webdriver executable. Do they reside in your working directory (D:\Data\_WebDriver\)? If that's isn't the cause, then I would guess some form of rights issue.

Edit: Another thing to check is your AV software, which could be detecting AutoIt and preventing it from launching the other executable.

The Webdriver executables resided in C:\Program Files (x86)\AutoIt\Include so I copied them to the working directory and that program ran OK. I then copied the executable to the working directories of my own programs. They still failed to work but there was a crucial difference in the console report:

Instead of "_WD_Startup ==> General Error: Error launching web driver!", I got "__WD_Post ==> Webdriver Exception: {"value":{"error":"session not created","message":"session not created: This version of ChromeDriver only supports Chrome version 91\nCurrent browser version is 93.0.4577.63 with ... "

Downloaded the right version and everything is hunky dory. That's the first time ever that Chrome's automatic, behind-the-scenes, updating has caused me a problem!

Many thanks for your reply, I just didn't know where to start looking. Your clue this time, and those in the past, have been invaluable.

[Danp2]

On 9/9/2021 at 4:23 AM, EagerLearner said:

Just came across an old thread by chance while trying to learn how to use WD, a user mentioned the new tab function wouldn't work after closing a tab and @Danp2 replied with new code that worked. As far as I can tell, this updated code was never integrated into the current release of WD on Git. Wanted to suggest the update. If I'm wrong or doing something wrong, please let me know as I am still very very new to programming in general.

I took another look at this and I think the issue is more involved than just _WD_NewTab. AFAICS, any attempt to interact with the webdriver console will fail because the active browsing context is invalid after the user manually closes the tab.

There's an active issue discussing this in the Github repo for anyone that wants to follow along or participate.

[Lunifrw]

I'm still quite new to AutoIT, and I need some food for thought on how I can read out a form field with a text and save it in a string.

Here is the HTML that would need to be read out:

 

<input type="email" class="whsOnd zHQkBf" jsname="YPqjbf" autocomplete="username" spellcheck="false" tabindex="0" aria-label="Nutzername" name="Username" autocapitalize="none" id="username" dir="ltr" data-initial-dir="ltr" data-initial-value=">>>>Diesen Inhalt möchte ich in einen String speichern<<<<" badinput="false">

[Danp2]

@Lunifrw Have you tried running wd_demo? You should be able to review it for some ideas. Post your code so that we can see what you've tried thus far.

P.S. See the following link for the proper way to post code on the forum

 

[Lunifrw]

 

Sorry, yesterday I was online on my mobile phone and I couldn't insert code.

 

As far as it works, the code has already tried to download the complete source code with "_WD_GetSource" and to filter out the content of the form with "StringRegExp", but isn't it also possible with the WebDriver?

 

The e-mail address is composed of the first name and surname and I would like to save the form of e-mail in the string $ sEmail.

 

Sorry about my English, I work with a translator because I'm from Germany.

 

expandcollapse popup

#include "wd_core.au3"
#include "wd_helper.au3"

Local $sDesiredCapabilities, $sSession, $sElement
Local $sVorname = 'xxx'
Local $sNachname = 'xxx'
Local $sEmail
SetupChrome()
_WD_Startup()
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, "http://exempel-adresse.de")



$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[@id='firstName']")
_WD_ElementAction($sSession, $sElement, 'value', $sVorname)
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//span[@class='whsOnd zHQkBf']")
_WD_ElementAction($sSession, $sElement, 'click')
_WD_LoadWait($sSession, 1000)


$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[@id='lastName']")
_WD_ElementAction($sSession, $sElement, 'value', $sNachname)
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//span[@class='whsOnd zHQkBf']")
_WD_ElementAction($sSession, $sElement, 'click')
_WD_LoadWait($sSession)



Func SetupChrome()
_WD_Option('Driver', 'chromedriver/chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args":["user-data-dir=C:\\Users\\' & @UserName & '\\AppData\\Local\\Google\\Chrome\\User Data\\", "--profile-directory=Default"], "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}'
                     # = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args":["start-maximized","disable-infobars","user-data-dir=C:\\Users\\' & @UserName & '\\AppData\\Local\\Google\\Chrome\\User Data\\", "--profile-directory=Default"] }}}}'
EndFunc

 

[Danp2]

@LunifrwThis code is from wd_demo --

$sValue1 = _WD_ElementAction($sSession, $sElement, 'property', 'value')
    $sValue2 = _WD_ElementAction($sSession, $sElement, 'value')

Each line shows a different method of retrieving an element's value.

Does that help?

[Lunifrw]

now that I see it that way it seems logical to me. that helped me thank you

[ThePoro]

Hello, 

I am having a problem with extract this part from Twitter with Webdriver UDF. I only want the part [unretweet] of this one below. Then with what command do I achieve it? 

Besides, I also want to extract href from the website too.

<div aria-expanded="false" aria-haspopup="menu" aria-label="126 Retweets. Retweeted" role="button" tabindex="0" class="css-18t94o4 css-1dbjc4n r-1777fci r-bt1l66 r-1ny4l3l r-bztko3 r-lrvibr" data-testid="unretweet"></div>

Its xPath = //*[@id="id__69y1m4etyv"]/div[2]/div

Many thanks!

Edited September 17, 2021 by ThePoro

[Danp2]

@ThePoroPlease take a moment and redefine the issue so that a literal thinker such as myself can understand. 😅

For instance --

• Are you wanting to retrieve the data-testid attribute regardless of its contents?
• What href is it that you want to extract? I imagine there's more than one on the site, so how are we to know which one you want?

Finally, post some code showing examples of what you've tried thus far.

Regards,

DanP

[ThePoro]

2 hours ago, Danp2 said:

@ThePoroPlease take a moment and redefine the issue so that a literal thinker such as myself can understand. 😅

For instance --

• Are you wanting to retrieve the data-testid attribute regardless of its contents?
• What href is it that you want to extract? I imagine there's more than one on the site, so how are we to know which one you want?

Finally, post some code showing examples of what you've tried thus far.

Regards,

DanP

Worked around for a moment, I finally understand how to use attribute with _WD_ElementAction. But now I have a problem that I want to Find Element with aforementioned attribute (data-testid) because of target's xPath isn't constant.

What should I do with this attribute = "Tweet" only? or is there any solution to remove the id_* part in xPath? Thank you so much!

//*[@id="id__n5y49tq3pva"]/div[2]/div

 

[seadoggie01]

@ThePoro There are lots of ways to build X-Paths, there's no need to include an ID if you don't want to. I'm guessing that you got that X-path by right clicking the Element in the dev console and selecting 'Copy XPath' or something similar. Note that you probably have another entry like "Copy Full XPath", which builds a super literal path to the element. It might look something like this: "/html/body/div[3]/div[2]/div/div/div/div/div/div/div/div/div/div/div/div/div/div[2]/div/div/div/div[2]/div[1]/table/tbody[1]/tr[2]". Ugly, right?

Alternately, you can try to build an XPath yourself. Here's a great resource for that: Xpath cheatsheet (devhints.io). Without seeing the full HTML, we can't write one for you, and honestly, I doubt anyone is interested in doing that anyways  You might try something like getting a parent's ID and walking down from there, or using something similar to //div[@data-testid="unretweet"] which means "get a div where data-testid is equal to unretweet"

[Danp2]

@ThePoroYou'll have to come up with the correct xpath. Have you tried something like this?

//div[@data-testid="Tweet"]

FWIW, I use the SelectorsHub add-in for determining xpaths.

[ThePoro]

  @Danp2 @seadoggie01 Thanks for all your helps. It actually works very well.

By the way, Do we have a way to run multiple browser profiles at the same time and let them do their automation programs simultaneously ?. I searched but unfortunately, I couldn't find any.

And, If I got an Array of Element ID from _WD_FindElement, now I want to know what are their ID. So how can I do that?

$ElementIDs = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//div[@data-testid="retweet"]',"",True)

It could be something like this id="id__n5y49tq3pva"

Edited September 18, 2021 by ThePoro

[seadoggie01]

Working with arrays: https://www.autoitscript.com/wiki/Arrays

16 hours ago, ThePoro said:

multiple browser profiles

"I do not think it means what you think it means". A profile is based around a user, so you might have multiple profiles if you, your mom, and your dad all use the browser. You can have multiple of these running, but I think it's (usually) silly to use them. Edit: in your script... elsewhere makes sense.

You can have multiple instances (called a session) of the browser running. This is like right clicking your browser and selecting "New Window". To do this, simply call _WD_CreateSession twice in a script, or once in two scripts. Be wary of calling _WD_Shutdown if they use the same driver instance (probably called a process?), however, because this can cause all sessions running with that process to be closed.

Finally, check out the WebDriver wiki: https://www.autoitscript.com/wiki/WebDriver. Great stuff there and will answer a bunch of questions you have!

Edited September 18, 2021 by seadoggie01

[seadoggie01]

Is  <URL>/<Port>/sessions an undocumented feature? When calling it, it gets an array of [capabilities, session ids]

I wrote a function to test it out and it seems to work fine in Chrome + Edge. I'd find this useful as I often use the same driver for two or more scripts, which makes calling _WD_Shutdown difficult

Spoiler

Func _WD_GetAllSessions()
    Local Const $sFuncName = "_WD_GetAllSessions"
    Local $sResponse = __WD_Get($_WD_BASE_URL & ":" & $_WD_PORT & "/sessions")
    Local $iErr = @error, $oValues = Null

    If $iErr = $_WD_ERROR_Success Then
        Local $oJSON = Json_Decode($sResponse)
        $oValues = Json_Get($oJSON, "[value]")

        If UBound($oValues) > 0 Then

            Local $aElements[UBound($oValues)][2]
            Local $iRow

            For $oValue In $oValues
                $aElements[$iRow][0] = Json_Get($oValue, "[capabilities]")

                $aElements[$iRow][0] = Json_Encode($aElements[$iRow][0])
                $aElements[$iRow][1] = Json_Get($oValue, "[id]")
                $iRow += 1
            Next
        EndIf

    EndIf

    If $_WD_DEBUG = $_WD_DEBUG_Info Then
        __WD_ConsoleWrite($sFuncName & ': ' & $sResponse & @CRLF)
    EndIf

    If $iErr Then
        Return SetError(__WD_Error($sFuncName, $iErr, "HTTP status = " & $_WD_HTTPRESULT), $_WD_HTTPRESULT, 0)
    EndIf

    Return SetError($_WD_ERROR_Success, $_WD_HTTPRESULT, $aElements)
EndFunc

 

[Danp2]

2 hours ago, seadoggie01 said:

Is  <URL>/<Port>/sessions an undocumented feature?

Must be a "Chromium" browser thing because it's not in the W3C specs and Geckodriver doesn't work with that endpoint.