WebDriver UDF - Help & Support (II)

[crazycrash]

Hi Danp2

I'm trying to get a parents element of the previously found elements in a loop. However I'm failing to use the _WD_FindElement function as it seems only the session can be used but not an element itself. Is that correct? Thanks for some guidance! (The xpath tested fine)

$postCommentElement = _WD_FindElement($aElements[$i], $_WD_LOCATOR_ByXPath, "../../../../div[@id='js_1al']//div", "", False)

__WD_Post: URL=HTTP://127.0.0.1:4444/session/c2eb04f7-af76-4bba-a2e1-50e48e3c974e/element; $sData={"using":"xpath","value":"/../../../../div[@id='js_1al']//div"}
__WD_Post: StatusCode=404; ResponseText={"value":{"error":"invalid session id","message":"Got unexpected session id c2eb04f7-af76-4bba-a2e1-...

 

Edited July 8, 2020 by crazycrash

[Danp2]

@crazycrash It should be obvious, but you can't use an element in place of the session ID. You may be able to accomplish your goal by using the previously found element for the $sStartElement parameter, but I haven't tried to do this to receive an ancestor of an element.

If that doesn't work, then you'll need to provide an actual example (website, short script) that we can using for testing.

[crazycrash]

I'm trying to get the matching text of a blog post to an image and extract it as text. Sometimes there are several images in one post. Maybe you have a better idea on how to achieve this? For example:

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error = $_WD_ERROR_Success Then
    ConsoleWrite("Starting extraction: " & @CRLF)
    _WD_Navigate($sSession, "https://www.facebook.com/AstrophysicsAndAstronomy/")
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//a[@rel='theater']", "", True) ; get image elements
    For $i = 0 To UBound($aElements, 1) - 1
        $limage = _WD_ElementAction($sSession, $aElements[$i], 'attribute', 'data-ploi') ; get image links
        If StringLen($limage) <> 0 Then ; if link found
            $CommentElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "/../../../../../../div[contains(@class, 'userContent')]/div", $aElements[$i], False) ; get matching comment from post
            ;$CommentText = $postCommentElement.gettext()
            ;MsgBox(0, "", $CommentText)
        EndIf
        Sleep(350)
    Next
EndIf
_WD_DeleteSession($sSession)
_WD_Shutdown()

This now gives:

__WD_Post: StatusCode=404; ResponseText={"value":{"error":"no such element","message":"Unable to locate element: /../../../../../../div[cont...

as it is only the starting element but not the element from which the xpath starts from.

Thanks

Edited July 8, 2020 by crazycrash

[Danp2]

@crazycrash Here's a revised version that works AFAICS --

expandcollapse popup

#include "wd_core.au3"
#include "wd_helper.au3"

Local $sDesiredCapabilities, $sSession, $sElement
Local $sImageXpath = "//a[@rel='theater' and @data-ploi]"
Local $sContentXpath = "//ancestor::div[@class='_3x-2']/preceding-sibling::div[contains(@class,'userContent')]"

SetupChrome()
_WD_Startup()

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error = $_WD_ERROR_Success Then
    ConsoleWrite("Starting extraction: " & @CRLF)
    _WD_Navigate($sSession, "https://www.facebook.com/AstrophysicsAndAstronomy/")
    _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, $sImageXpath, 2000)
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sImageXpath, Default, True) ; get image elements
    For $i = 0 To UBound($aElements, 1) - 1
        $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sContentXpath) ; get user content elements

        If @error = $_WD_ERROR_Success Then
            $sComment = _WD_ElementAction($sSession, $sElement, 'attribute', 'innerText')
            ConsoleWrite($sComment & @crlf & "===============" & @CRLF)
            Sleep(350)
        EndIf
    Next

EndIf

_WD_DeleteSession($sSession)
_WD_Shutdown()

Func SetupChrome()
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}'
EndFunc

 

[ngocthang26]

<input class="_58al _2gnb" type="text" aria-invalid="false" aria-labelledby="js_1b" placeholder="Enter a name..." value="" xpath="1">

When I use this code

_WD_WaitElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@class='_58al _2gnb']")
$sElement = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@class='_58al _2gnb']")
_WD_ElementAction($sSession, $sElement, 'value',"test")

Don't set value

But i try code js

document.getElementsByClassName("_58al _2gnb")[1].value="dasd";

Can I identify false xpath. How do click element with class have [0] [1] [2] . I try change postion //input[@class='_58al _2gnb'[0]], //input[@class='_58al _2gnb\[0]'], //input[@class='_58al _2gnb[0]']. Anything fales

__WD_Post: URL=HTTP://127.0.0.1:9515/session/9a5b499e43e9ce859743d05f1739dd70/element; $sData={"using":"xpath","value":"//input[@class='_58al _2gnb'][0]"}
__WD_Post: StatusCode=404; ResponseText={"value":{"error":"no such element","message":"Unable to locate element{\"method\"\"xpath\",\"selector\"\"//input[@class='_58al _2gnb'][0]\"}(Session infochrome=83.0.4103.61)","stacktrace":"Backtrace:\n\tOrdinal0 [0x00C03CF0+867568]\n\tOrdinal0 [0x00BADB7D+514941]\n\tOrdinal0 [0x00B7B367+308071]\n\tOrdinal0 [0x00B49425+103461]\n\tOrdinal0 [0x00B63B70+211824]\n\tOrdinal0 [0x00B5AEC0+175808]\n\tOrdinal0 [0x00B6278D+206733]\n\tOrdinal0 [0x00B5AD2B+175403]\n\tOrdinal0 [0x00B43740+79680]\n\tOrdinal0 [0x00B44CFC+85244]\n\tOrdinal0 [0x00B44C50+85072]\n\tOrdinal0 [0x00C0E62A+910890]\n\tOrdinal0 [0x00BB8993+559507]\n\tOrdinal0 [0x00BB8BA3+560035]\n\tOrdinal0 [0x00BB8C8C+560268]\n\tOrdinal0 [0x00C11907+923911]\n\tOrdinal0 [0x00BB87FF+559103]\n\tOrdinal0 [0x00BC306E+602222]\n\tOrdinal0 [0x00BCEA3B+649787]\n\tOrdinal0 [0x00BCEBA5+650149]\n\tOrdinal0 [0x00BCDDA5+646565]\n\tBaseThreadInitThunk [0x751560C9+25]\n\tRtlGetAppContainerNamedObjectPath [0x771B7A94+228]\n\tRtlGetAppContainerNamedObjectPath [0x771B7A64+180]\n"}}
_WD_FindElement: {"value":{"error":"no such element","message":"Unable to locate element{\"method\"\"xpath\",\"selector\"\"//input[@class='_58al _2gnb'][0]\"}(Session infochrome=83.0.4103.61)","stacktrace":"Backtrace:\n\tOrdinal0 [0x00C03CF0+867568]\n\tOrdinal0 [0x00BADB7D+514941]\n\tOrdinal0 [0x00B7B367+308071]\n\tOrdinal0 [0x00B49425+103461]\n\tOrdinal0 [0x00B63B70+211824]\n\tOrdinal0 [0x00B5AEC0+175808]\n\tOrdinal0 [0x00B6278D+206733]\n\tOrdinal0 [0x00B5AD2B+175403]\n\tOrdinal0 [0x00B43740+79680]\n\tOrdinal0 [0x00B44CFC+85244]\n\tOrdinal0 [0x00B44C50+85072]\n\tOrdinal0 [0x00C0E62A+910890]\n\tOrdinal0 [0x00BB8993+559507]\n\tOrdinal0 [0x00BB8BA3+560035]\n\tOrdinal0 [0x00BB8C8C+560268]\n\tOrdinal0 [0x00C11907+923911]\n\tOrdinal0 [0x00BB87FF+559103]\n\tOrdinal0 [0x00BC306E+602222]\n\tOrdinal0 [0x00BCEA3B+649787]\n\tOrdinal0 [0x00BCEBA5+650149]\n\tOrdinal0 [0x00BCDDA5+646565]\n\tBaseThreadInitThunk [0x751560C9+25]\n\tRtlGetAppContainerNamedObjectPath [0x771B7A94+228]\n\tRtlGetAppContainerNamedObjectPath [0x771B7A64+180]\n"}}
_WD_FindElement ==> No match: HTTP status = 404
_WD_WaitElement ==> Timeout

 

Thanks you so much

Edited July 9, 2020 by ngocthang26

[Math43]

Hi,

I'm trying to dismiss an alert popup in Chrome but I'm not sure how to do. I've tried with :

_WD_Alert($sSession, 'dismiss')

but i've got an error in return:

__WD_Post: URL=HTTP://127.0.0.1:57053/session/4b0858233f6c2266e3f3eaaa70b043fb/alert/dismiss; $sData={}
__WD_Post: StatusCode=404; ResponseText={"value":{"error":"no such alert","message":"no such alert\n  (Session info: chrome=83.0.4103.116)",...
_WD_Alert: {"value":{"error":"no such alert","message":"no such alert\n  (Session info: chrome=83.0.4103.116)","stacktrace":"Backtrace:\n\tOrdinal0 [0x01369563+2725219]\n\tOrdinal0 [0x01268551+1672529]\n\tOrdinal0 [0x01150202+524802]\n\tOrdinal0 [0x0114D651+513617]\n\tOrdinal0 [0x010D4B65+19301]\n\tOrdinal0 [0x010FAAFA+174842]\n\tOrdinal0 [0x010D4704+18180]\n\tOrdinal0 [0x010FA94B+174411]\n\tOrdinal0 [0x010E2528+75048]\n\tOrdinal0 [0x010E35A0+79264]\n\tOrdinal0 [0x010E3539+79161]\n\tOrdinal0 [0x0127D607+1758727]\n\tGetHandleVerifier [0x01486546+1050150]\n\tGetHandleVerifier [0x01486291+1049457]\n\tGetHandleVerifier [0x014910D7+1094071]\n\tGetHandleVerifier [0x01486B46+1051686]\n\tOrdinal0 [0x01275B06+1727238]\n\tOrdinal0 [0x0127EB7B+1764219]\n\tOrdinal0 [0x0127ECE3+1764579]\n\tOrdinal0 [0x01294C05+1854469]\n\tBaseThreadInitThunk [0x755962C4+36]\n\tRtlSubscribeWnfStateChangeNotification [0x77BD0969+1081]\n\tRtlSubscribeWnfStateChangeNotification [0x77BD0934+1028]\n"}}
_WD_Alert ==> No alert present: {"value":{"error":"no such alert","message":"no such alert\n  (Session info: chrome=83.0.4103.116)","stacktrace":"Backtrace:\n\tOrdinal0 [0x01369563+2725219]\n\tOrdinal0 [0x01268551+1672529]\n\tOrdinal0 [0x01150202+524802]\n\tOrdinal0 [0x0114D651+513617]\n\tOrdinal0 [0x010D4B65+19301]\n\tOrdinal0 [0x010FAAFA+174842]\n\tOrdinal0 [0x010D4704+18180]\n\tOrdinal0 [0x010FA94B+174411]\n\tOrdinal0 [0x010E2528+75048]\n\tOrdinal0 [0x010E35A0+79264]\n\tOrdinal0 [0x010E3539+79161]\n\tOrdinal0 [0x0127D607+1758727]\n\tGetHandleVerifier [0x01486546+1050150]\n\tGetHandleVerifier [0x01486291+1049457]\n\tGetHandleVerifier [0x014910D7+1094071]\n\tGetHandleVerifier [0x01486B46+1051686]\n\tOrdinal0 [0x01275B06+1727238]\n\tOrdinal0 [0x0127EB7B+1764219]\n\tOrdinal0 [0x0127ECE3+1764579]\n\tOrdinal0 [0x01294C05+1854469]\n\tBaseThreadInitThunk [0x755962C4+36]\n\tRtlSubscribeWnfStateChangeNotification [0x77BD0969+1081]\n\tRtlSubscribeWnfStateChangeNotification [0x77BD0934+1028]\n"}}

 

[Danp2]

@ngocthang26 Telling us that something "doesn't work" without providing any further details isn't going to get you very far. I'm guessing that _WD_FindElement returned an error, but you didn't show us those results.

Not sure how to answer your question regarding classes with [0],[1], etc because again you haven't provided enough details. Show us the HTML for the element that you are attempting to locate. I suspect that you are trying to deal with an array where there are multiple matching controls, but there's no way to know with the limited information you've posted thus far.

Help us by providing sufficient details so that we can help you.

 

[ngocthang26]

3 minutes ago, Danp2 said:

@ngocthang26 Telling us that something "doesn't work" without providing any further details isn't going to get you very far. I'm guessing that _WD_FindElement returned an error, but you didn't show us those results.

Not sure how to answer your question regarding classes with [0],[1], etc because again you haven't provided enough details. Show us the HTML for the element that you are attempting to locate. I suspect that you are trying to deal with an array where there are multiple matching controls, but there's no way to know with the limited information you've posted thus far.

Help us by providing sufficient details so that we can help you.

 

Yes sorry all. I will provide more infomation

[Danp2]

@Math43 That function is designed to handle simple user prompts, which I'm guessing that doesn't apply to your situation. A detailed description of this webdriver functionality can be found here.

There's likely a way to handle this with other WD UDF functions. Try examining this popup in the browser's Developer Tools. What does the associated HTML look like? Does it used frames? Etc.

[crazycrash]

13 hours ago, Danp2 said:

@crazycrash Here's a revised version that works AFAICS --

expandcollapse popup

#include "wd_core.au3"
#include "wd_helper.au3"

Local $sDesiredCapabilities, $sSession, $sElement
Local $sImageXpath = "//a[@rel='theater' and @data-ploi]"
Local $sContentXpath = "//ancestor::div[@class='_3x-2']/preceding-sibling::div[contains(@class,'userContent')]"

SetupChrome()
_WD_Startup()

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error = $_WD_ERROR_Success Then
    ConsoleWrite("Starting extraction: " & @CRLF)
    _WD_Navigate($sSession, "https://www.facebook.com/AstrophysicsAndAstronomy/")
    _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, $sImageXpath, 2000)
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sImageXpath, Default, True) ; get image elements
    For $i = 0 To UBound($aElements, 1) - 1
        $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sContentXpath) ; get user content elements

        If @error = $_WD_ERROR_Success Then
            $sComment = _WD_ElementAction($sSession, $sElement, 'attribute', 'innerText')
            ConsoleWrite($sComment & @crlf & "===============" & @CRLF)
            Sleep(350)
        EndIf
    Next

EndIf

_WD_DeleteSession($sSession)
_WD_Shutdown()

Func SetupChrome()
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}'
EndFunc

 

Thanks, just learned a few things. Is my understanding correct, that a relative path then relates to the previously found element?

Also, your example works great except for:

$sComment = _WD_ElementAction($sSession, $sElement, 'attribute', 'innerText')

  I get the following response:

__WD_Post: URL=HTTP://127.0.0.1:4444/session/72f14809-fdb0-4e77-9efd-c3fff9fb555d/element; $sData={"using":"xpath","value":"//ancestor::div[@class='_3x-2']/preceding-sibling::div[contains(@class,'userContent')]"}
__WD_Post: StatusCode=200; ResponseText={"value":{"element-6066-11e4-a52e-4f735466cecf":"26bb4535-a040-4a4e-8d44-43d64eb4b67d"}}...
_WD_FindElement: {"value":{"element-6066-11e4-a52e-4f735466cecf":"26bb4535-a040-4a4e-8d44-43d64eb4b67d"}}
__WD_Get: URL=HTTP://127.0.0.1:4444/session/72f14809-fdb0-4e77-9efd-c3fff9fb555d/element/26bb4535-a040-4a4e-8d44-43d64eb4b67d/attribute/innerText
__WD_Get: StatusCode=200; $iResult = 0; $sResponseText={"value":null}...
_WD_ElementAction: {"value":null}...

===============

innerHtml also doesn't work. Could it be due to firefox vs chrome?

Thanks

Edited July 9, 2020 by crazycrash

[Danp2]

12 minutes ago, crazycrash said:

Could it be due to firefox vs chrome?

Appears to be so, although I don't understand why. Guessing that Firefox isn't locating the correct element for some reason. I'll have to do some further investigation to find out what's going on.

[crazycrash]

Thanks. The xpath

//ancestor::div[@class='_3x-2']/preceding-sibling::div[contains(@class,'userContent')]

works fine in Firefox if added to the xpath of the image in chropath. And the correct element seems to be returned for the images.

Edited July 9, 2020 by crazycrash

[Math43]

 

2 hours ago, Danp2 said:

@Math43 That function is designed to handle simple user prompts, which I'm guessing that doesn't apply to your situation. A detailed description of this webdriver functionality can be found here.

There's likely a way to handle this with other WD UDF functions. Try examining this popup in the browser's Developer Tools. What does the associated HTML look like? Does it used frames? Etc.

 

Thank for you answer. In fact this popup seem to be generated by the browser and not by the page, i'm unable to find anything in the dev tool and there is no html or associated.

[Danp2]

@Math43 Unsure if you can bypass that prompt, but you may want to do some research on that.

Maybe this will help -- https://communities.vmware.com/thread/620083

[Math43]

Just now, Danp2 said:

@Math43 Unsure if you can bypass that prompt, but you may want to do some research on that.

Maybe this will help -- https://communities.vmware.com/thread/620083

I've just found a way with a Chrome's GPO for Windows by using URLWhitelist option.

Registry tweak is:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome\URLWhitelist]
"1"="vmware-plugin://*"

Anyway, thank again for you help Danp2 !

 

[nooneclose]

@Danp2 How do I connect to an existing browser session with webdriver? 

 

I remember seeing a form post about Firefox being able to do this with webdriver but I cannot find that post again.

Do you have the lines of code needed to connect to an existing open browser? (it doesn't have to be firefox, chrome would be great also)

[water]

Please have a look at the FAQ in the wiki: https://www.autoitscript.com/wiki/WebDriver

[nooneclose]

@water I did what you suggested and found this code:

_WD_Option('Driver', 'C:\Path\to\the\executable\geckodriver.exe')
_WD_Option('DriverParams', '--log trace --connect-existing  --marionette-port 2828')
_WD_Option('Port', 4444)

However, I am getting this error.

1594316263915   geckodriver     DEBUG   Listening on HIDDEN
1594316264342   webdriver::server       DEBUG   -> POST /session {"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}
1594316264343   webdriver::command      WARN    You are using deprecated legacy session negotiation patterns (desiredCapabilities/requiredCapabilities), see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities#Legacy
1594316264351   geckodriver::marionette DEBUG   Waiting 60s to connect to browser on HIDDEN

I think my line below is outdated: @Danp2

$sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'

 

Edited July 9, 2020 by nooneclose

[Danp2]

@nooneclose Yep... use this format instead --

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}'

Adjust as needed.

[nooneclose]

@Danp2 Thank you for getting back with me.

I switched the format like you said. Now I get this error:

1594318965224   geckodriver     DEBUG   Listening on Hidden
1594318965651   webdriver::server       DEBUG   -> POST /session {"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}
1594318965654   geckodriver::capabilities       DEBUG   Trying to read firefox version from ini files
1594318965656   geckodriver::capabilities       DEBUG   Found version 78.0.1
1594318965659   geckodriver::marionette DEBUG   Waiting 60s to connect to browser on Hidden

_WD_IsLatestRelease: False
_WD_IsLatestRelease ==> Success
_WDStartup: OS: WIN_10 WIN32_NT 18362 
_WDStartup: AutoIt: 3.3.14.5
_WDStartup: WD.au3: 0.2.0.9 (Update available)
_WDStartup: WinHTTP:    1.6.4.1
_WDStartup: Driver: geckodriver.exe
_WDStartup: Params: --log trace --connect-existing  --marionette-port 2828
_WDStartup: Port:   4444
__WD_Post: URL=HTTP://HIDDEN:4444/session; $sData={"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}
__WD_Post: StatusCode=0; ResponseText=0...
__WD_Post ==> Send / Recv error
_WD_CreateSession: 0
_WD_CreateSession ==> Webdriver Exception: HTTP status = 0
Coping Data at line number: 305__WD_Post: URL=HTTP://HIDDEN:4444/session//element; $sData={"using":"xpath","value":"//a[@id='m185c951-tab_anchor']"}
__WD_Post: StatusCode=405; ResponseText=HTTP method not allowed...
_WD_FindElement: HTTP method not allowed

 

Edited July 9, 2020 by nooneclose
more info