WebDriver UDF - Help & Support (II)

[Danp2]

As the WebDriver UDF - Help & Support thread has grown too big, I started a new one.

The original thread can be found here.

Edited December 10, 2019 by Danp2

[jbnet]

I would like to know if it is possible to access these network resources after loading the page in firefox as shown in the image with webdriver I tried sending javascript command, sending header requests but with my little experience I could not get this data as the image, but I want it retrieve the data as an example in the image, this request list, but I don't know how to get it. I thank you for this work I like this udf very much.

[Danp2]

Hi @jbnet,

I'm not sure that I understand what you are trying to accomplish. Can you explain your "use case"? Also, it might help if you showed us what you've tried thus far.

Dan

[jbnet]

8 hours ago, Danp2 said:

Olá @jbnet ,

Não sei ao certo o que você está tentando realizar. Você pode explicar seu "caso de uso"? Além disso, pode ajudar se você nos mostrou o que tentou até agora.

Dan

I am wanting to get all xhr data from the header. I want to get all URL lists, but the javascript code that I tested gets only the answers but I can't get the list of URLs. For example, upon entering the site, several requests are generated. I want to capture the urls of the generated requests, all the list, as I showed marked in the image, but I don't know how to get the whole list. The javascript example I sent doesn't have all the information, like the URLs. In the header, I want to get requests not only the answers, ie all the information cited in the browser.

_WD_Startup()

$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, "https://google.com")

$testeJavaScript = _WD_ExecuteScript($sSession,"function getHeadersAjax(url) { "& _
  " var r = new XMLHttpRequest();"& _
   "r.open('HEAD', url, false);"& _
   "r.send(null);"& _
   "return {"& _
   "   status: r.status, "& _
   "   statusText: r.statusText, "& _
   "   referrer: document.referrer,"& _
   "   rawheaders: r.getAllResponseHeaders(),"& _
    "};"& _
"} return getHeadersAjax(document.location.href); ")
MsgBox(0, "Infor", $testeJavaScript)


_WD_DeleteSession($sSession)
_WD_Shutdown()

Another situation is how to get information from the firefox console, retrieve this information for autoit, is it possible?

 

Edited December 15, 2019 by jbnet

[Danp2]

1 hour ago, jbnet said:

I want to capture the urls of the generated requests, all the list, as I showed marked in the image, but I don't know how to get the whole list.

Why? How would you use this information if you could capture it?

P.S. Not sure that this is possible without using a proxy

[jbnet]

1 hour ago, Danp2 said:

Por quê? Como você usaria essas informações se pudesse capturá-las?

PS Não tenho certeza se isso é possível sem usar um proxy

I use webdriver to capture videos, searching for urls, I found a site where the url information to download the m3u8 file is in the header with this list I use fmpeg to download, inspecting the page the way I showed you in the image II find this url, that's why I would like to capture it. but if it is not possible, if I had to manipulate Firefox Extensions as downloadhelper capturing videos would also solve.

[RandalBY]

Hi, @Danp2

Is there any documentation for functions of your UDF (like main doc's)? I found only list of functions on the 1st page without any details... 

[nguyenthaiytcc]

Hi @Danp2, I have a new problem :))
My computer is connected to 2 network cards, I will call network 1 and network 2.
I want the network option to use corresponding to DesCap, For example: Session 1 - DesiredCapabilities 1 - Network 1 and Session 2 - DesiredCapabilities 2 - Network 2.
Can I do this with Argument?
If not, how can I do this? Give me some keywords on this issue?

Edited December 16, 2019 by nguyenthaiytcc

[Danp2]

Hi @RandalBY

No, I haven't thus far created a help file for this UDF. There are a set of examples that have been gathered together in wd_demo.au3, which is currently the best way to understand the operation of various functions.

P.S. This is an opensource project, so anyone is free to write the docs if they get the urge. 😄

[Danp2]

@nguyenthaiytcc Sorry, but I'm not sure that I follow your post. Is "season" = "session"? And "Descap" is "DesiredCapabilities"?

Please restate your issue and be sure to explain why you need to control which network card is being used.

[nguyenthaiytcc]

Dear @Danp2, I am trying to access multiple accounts at the same time with multiple login sessions with multiple ip addresses as well as UAs. However, there are some websites that can detect MAC address, I thought I should use virtual network card to fake different MAC address. Is this possible?

[Danp2]

@nguyenthaiytcc No idea if this is possible and not something I'm planning to investigate. 😉

[RandalBY]

1 hour ago, nguyenthaiytcc said:

Dear @Danp2, I am trying to access multiple accounts at the same time with multiple login sessions with multiple ip addresses as well as UAs. However, there are some websites that can detect MAC address, I thought I should use virtual network card to fake different MAC address. Is this possible?

websites can't detect your MAC. But if you want, you can use routing settings in windows (google for "route windows")

[RandalBY]

@Danp2 Can you give any examples how to use $sStartElement option in _WD_FindElement? Or there is no sense to use it in real life?

[Danp2]

47 minutes ago, RandalBY said:

Can you give any examples how to use $sStartElement option in _WD_FindElement?

 

[J-Luc]

Hello All,

I have just discovered this UDF and tried to use it with Firefox with geckodriver.

_WD_Startup and _WD_CreateSession functions work fine to start the driver and open a browser window, but all the other commands return in error with "StatusCode=405; ResponseText=HTTP method not allowed"

Can anybody advice me ?

I also get a "_WD_CreateSession ==> Webdriver Exception" but I am not sure it is linked or not.

[water]

Welcome to AutoIt and the forum!

Could you please post the script you run and the full output of the console?

[J-Luc]

Sure, here is the script
 

#include "common\wd_core.au3"
Local $sDesiredCapabilities
SetupGecko()
_WD_Startup()
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, "http://sit")
ConsoleWrite("URL=" & _WD_Action($sSession, 'url') & @CRLF)
_WD_Shutdown()

Func SetupGecko()
_WD_Option('Driver', 'geckodriver.exe')
_WD_Option('DriverParams', '--log trace')
_WD_Option('Port', 4444)
$sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc

And output :

_WDStartup: OS:    WIN_10 WIN32_NT 16299
_WDStartup: AutoIt:    3.3.14.5
_WDStartup: WD.au3:    0.2.0.3
_WDStartup: WinHTTP:    1.6.4.1
_WDStartup: Driver:    geckodriver.exe
_WDStartup: Params:    --log trace
_WDStartup: Port:    4444
__WD_Post: URL=HTTP://127.0.0.1:4444/session; $sData={"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}
__WD_Post: StatusCode=200; ResponseText={"value":{"sessionId":"c09c0114-40f0-4741-8474-1c5856efb076","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"71.0","javascriptEnabled":true,"moz:accessibilityChecks":false,"moz:buildID":"20191202093317","moz:geckodriverVersion":"0.26.0","moz:headless":false,"moz:processID":7884,"moz:profile":"C:\\Users\\pb19348\\AppData\\Local\\Temp\\rust_mozprofileM0GXnv","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"nativeEvents":true,"pageLoadStrategy":"normal","platformName":"windows","platformVersion":"10.0","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
_WD_CreateSession: {"value":{"sessionId":"c09c0114-40f0-4741-8474-1c5856efb076","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"71.0","javascriptEnabled":true,"moz:accessibilityChecks":false,"moz:buildID":"20191202093317","moz:geckodriverVersion":"0.26.0","moz:headless":false,"moz:processID":7884,"moz:profile":"C:\\Users\\pb19348\\AppData\\Local\\Temp\\rust_mozprofileM0GXnv","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"nativeEvents":true,"pageLoadStrategy":"normal","platformName":"windows","platformVersion":"10.0","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
_WD_CreateSession ==> Webdriver Exception
__WD_Post: URL=HTTP://127.0.0.1:4444/session//url; $sData={"url":"http://sit"}
__WD_Post: StatusCode=405; ResponseText=HTTP method not allowed
_WD_Navigate: HTTP method not allowed
__WD_Get: URL=HTTP://127.0.0.1:4444/session//url
__WD_Get: StatusCode=405; $iResult = 0; $sResponseText=HTTP method not allowed...
_WD_Action: HTTP method not allowed
URL=0

 

[Danp2]

16 minutes ago, J-Luc said:

_WDStartup: WinHTTP:    1.6.4.1

Please update to the latest source available from here on Github and then retest your script.

[RandalBY]

Is it possible to connect to an existing browser window that was opened by a script in a previous session?