WebDriver UDF - Help & Support (IV)

[Danp2]

@rcmaehlI would suggest adding this line before calling _WD_Startup --

$_WD_Debug = $_WD_DEBUG_Full

Then run the script and post the complete contents of the Scite output panel.

[rcmaehl]

3 minutes ago, Danp2 said:

@rcmaehlI would suggest adding this line before calling _WD_Startup --

$_WD_Debug = $_WD_DEBUG_Full

Then run the script and post the complete contents of the Scite output panel.

 

_WD_Option ==> Success [0] : Parameters:   Option=Driver   Value=chromedriver.exe
_WD_Option ==> Success [0] : Parameters:   Option=Port   Value=9515
_WD_Option ==> Success [0] : Parameters:   Option=DriverParams   Value=--verbose --log-path="C:\Users\rcmaehl\Documents\GitHub\MSEdgeRedirect\Includes\au3WebDriver\chrome.log"
_WD_Startup: OS:    WIN_10 WIN32_NT 19044 
_WD_Startup: AutoIt:    3.3.16.0
_WD_Startup: Webdriver UDF: 0.10.1
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --verbose --log-path="C:\Users\rcmaehl\Documents\GitHub\MSEdgeRedirect\Includes\au3WebDriver\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --verbose --log-path="C:\Users\rcmaehl\Documents\GitHub\MSEdgeRedirect\Includes\au3WebDriver\chrome.log" 
_WD_Startup ==> Success [0]
__WD_Post: URL=HTTP://127.0.0.1:9515/session; Data={"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"debuggerAddress": "localhost:9222"}}}}
__WD_Post ==> Send / Recv error [6] : HTTP status = 0 ResponseText=WinHTTP request timed out before Webdriver
_WD_CreateSession ==> Webdriver Exception [10]
__WD_Post: URL=HTTP://127.0.0.1:9515/session//url; Data={"url":"http://yahoo.com"}
__WD_Post ==> Invalid session ID [15] : HTTP status = 404 ResponseText={"value":{"error":"invalid session id","message":"invalid session id","stacktrace":"Backtrace:\n\tOrdinal0 [0x002878B3+2193587]\n\tOrdinal0 [0x00220681+1771137]\n\tOrdinal0 [0x00134070+802928]\n\tOrdinal0 [0x00157260+946784]\n\tOrdinal0 [0x0017D6A0+1103520]\n\tOrdinal0 [0x0017B4F5+1094901]\n\tOrdinal0 [0x0017B098+1093784]\n\tOrdinal0 [0x001155C7+677319]\n\tOrdinal0 [0x00115B43+678723]\n\tOrdinal0 [0x00115F7A+679802]\n\tGetHandleVerifier [0x00530CB2+2738370]\n\tGetHandleVerifier [0x005221B8+2678216]\n\tGetHandleVerifier [0x003117AA+512954]\n\tGetHandleVerifier [0x00310856+509030]\n\tOrdinal0 [0x0022743B+1799227]\n\tOrdinal0 [0x00115357+676695]\n\tOrdinal0 [0x00114E18+675352]\n\tGetHandleVerifier [0x005542BC+2883276]\n\tBaseThreadInitThunk [0x7564FA29+25]\n\tRtlGetAppContainerNamedObjectPath [0x776D7A9E+286]\n\tRtlGetAppContainerNamedObjectPath [0x776D7A6E+238]\n\t(No symbol) [0x00000000]\n"}}
_WD_Navigate ==> Invalid session ID [15] : Parameters:   URL=http://yahoo.com

 

[Danp2]

@rcmaehlAnything useful in chrome.log? What is your Chrome version? Same for Chromedriver.

[rcmaehl]

2 hours ago, Danp2 said:

@rcmaehlAnything useful in chrome.log? What is your Chrome version? Same for Chromedriver.

Nothing useful.

[1660505185.251][INFO]: Starting ChromeDriver 104.0.5112.79 (3cf3e8c8a07d104b9e1260c910efb8f383285dc5-refs/branch-heads/5112@{#1307}) on port 9515
[1660505185.252][INFO]: Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.

Chrome version is: Version 104.0.5112.81 (Official Build) (64-bit)

I tried some directory changes and disabled Windows Firewall temporarily to try to resolve this issue with no change.

Oddly enough, Edge "works", but it opens a new browser window instead of letting me control the existing window.

 

[mLipok]

What means interactive:

Quote

1661334041520   webdriver::server       DEBUG   -> POST /session/54597147-bab9-4350-afdf-59030f6d3b4f/execute/sync {"script":"return document.readyState", "args":[]}
1661334041529   webdriver::server       DEBUG   <- 200 OK {"value":"interactive"}

 

Also i get few times uninitialized

EDIT 1:
 

Quote

1661334259026   webdriver::server       DEBUG   -> POST /session/7f888395-3bc5-489a-880b-0ce53aa770ca/execute/sync {"script":"return document.readyState", "args":[]}
1661334259029   webdriver::server       DEBUG   <- 200 OK {"value":"uninitialized"}

EDIT 2:

As so far I found:
https://stackoverflow.com/questions/42881021/how-to-interact-with-page-with-ready-status-as-interactive-in-selenium

https://stackoverflow.com/questions/56728656/what-is-the-correct-syntax-checking-the-readystate-of-a-website-in-selenium-pyt

EDIT 3:

And finally from here: https://www.w3schools.com/jsref/prop_doc_readystate.asp

Syntax

document.readyState

Return Value

Type	Description
String	The status of the current document: uninitialized - Has not started loading loading - Is loading loaded - Has been loaded interactive - Has loaded enough to interact with complete - Fully loaded

 

 

EDIT 4:

https://www.w3.org/TR/webdriver2/#dfn-waiting-for-the-navigation-to-complete

 

EDIT 5:
https://www.selenium.dev/documentation/webdriver/capabilities/shared/

The page load strategy queries the document.readyState as described in the table below:

Strategy	Ready State	Notes
normal	complete	Used by default, waits for all resources to download
eager	interactive	DOM access is ready, but other resources like images may still be loading
none	Any	Does not block WebDriver at all

Edited August 24, 2022 by mLipok

[Danp2]

@mLipokSeems like you've answered your own question. 😄

[mLipok]

10 minutes ago, Danp2 said:

@mLipokSeems like you've answered your own question. 😄

As I am curious ... It happen to me to answer my own questions quite often

[triple_N]

Hi @Danp2

First of all thanks for the effort of making this awesome and simple udf.

I able to implement it successfully but may I know whether the web driver(geckodriver.exe) cmd able to run in background or not?

My guess adding into 

_WD_Option('DriverParams', '--log trace')

but couldn't find the correct term.

I don't want my browser automate headless, only the web driver cmd to not be shown.

[Danp2]

@triple_NSee FAQ #2 on the wiki.

[SaeidN]

 

It's been one hour I'm looking everywhere to see how I can open a url (let's say www.yahoo.com) in chrome using webdriver UDF, and didn't find anything.

 

[Danp2]

@SaeidNHave you checked out the demo script (wd_demo)?

[SaeidN]

@Danp2

I have chromedriver and it starts, chrome window opens for a sec and closes. Then, it shows this:

> UpdateResult = False
> wd_demo.au3: _WD_Startup
> wd_demo.au3: _WD_CreateSession

! Error = 10 occurred on: Demo
! _WD_LastHTTPResult = 0
! _WD_GetSession =

 

Edited September 7, 2022 by SaeidN

[Danp2]

@SaeidNWe need to figure out why _WD_CreateSession is failing. Please rerun the script and change the debug level to Full before clicking the Run Demo button. Then please post the complete script output here (you can use the same method shown in how to post code).

[SaeidN]

@Danp2 Here's the full log.

expandcollapse popup

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" /ErrorStdOut "C:\Users\me\Documents\wd_demo.au3"    
_WD_Option ==> Success [0] : Parameters:   Option=errormsgbox   Value=False
_WD_Option ==> Success [0] : Parameters:   Option=OutputDebug   Value=False
_WD_GetBrowserPath ==> Success [0 / 0] : Parameters:    Browser=Chrome
_WD_GetBrowserVersion ==> Success [0 / 0] : Parameters:    Browser=Chrome
_WD_GetWebDriverVersion ==> Success [0 / 0] : Parameters:    Dir=C:\Users\me\Documents\    EXE=chromedriver.exe
__WD_GetLatestWebdriverInfo ==> Success [0 / 0]
_WD_UpdateDriver ==> Success [0 / 0] : DriverCurrent = 106.0.5249.21 : DriverLatest = 105.0.5195.52
> UpdateResult = False
_WD_Option ==> Success [0] : Parameters:   Option=Driver   Value=chromedriver.exe
_WD_Option ==> Success [0] : Parameters:   Option=Port   Value=9515
_WD_Option ==> Success [0] : Parameters:   Option=DriverParams   Value=--verbose --log-path="C:\Users\me\Documents\chrome.log"
_WD_CapabilitiesStartup: #148 : > {"capabilities":"{}"} > IsObj = 1
_WD_CapabilitiesAdd: #177 :      $key = alwaysMatch     $value1 = chrome     $value2 = 
_WD_CapabilitiesAdd: #205 :  $_WD_NOTATION__MATCHTYPE = [capabilities][alwaysMatch] $_WD_NOTATION__SPECIFICVENDOR = ["goog:chromeOptions"]
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd: #177 :      $key = w3c     $value1 = True     $value2 = 
_WD_CapabilitiesAdd: #257 #273 :      $key = w3c     $value1 = True     $value2 =     $s_Notation = [capabilities][alwaysMatch]["goog:chromeOptions"]["w3c"]   <<<<  True
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd: #177 :      $key = excludeSwitches     $value1 = enable-automation     $value2 = 
_WD_CapabilitiesAdd: #239 #273 :      $key = excludeSwitches     $value1 = enable-automation     $value2 =     $s_Notation = [capabilities][alwaysMatch]["goog:chromeOptions"]["excludeSwitches"][0]   <<<<  enable-automation
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesDump: JSON structure starts below: 986
{
    "capabilities":{
        "alwaysMatch":{
            "goog:chromeOptions":{
                "w3c":true,
                "excludeSwitches":[
                    "enable-automation"
                ]
            }
        }
    }
}
_WD_CapabilitiesDump: JSON structure ends above.
> wd_demo.au3: _WD_Startup
_WD_IsLatestRelease ==> Success [0] : True
"C:\Users\me\Documents\wd_core.au3" (1237) : ==> Unknown function name.:
Local $sWinHttpVer = __WinHttpVer()
Local $sWinHttpVer = ^ ERROR
>Exit code: 1

 

[Danp2]

A couple of potential issues --

3 minutes ago, SaeidN said:

_WD_UpdateDriver ==> Success [0 / 0] : DriverCurrent = 106.0.5249.21 : DriverLatest = 105.0.5195.52

If your Chrome version isn't 106.x, then that could explain the initial scenario where Chrome briefly opened, then closed. You need to run matching version of Chromedriver / Chrome (where at least the major version number is the same).

Quote

"C:\Users\me\Documents\wd_core.au3" (1237) : ==> Unknown function name.:
Local $sWinHttpVer = __WinHttpVer()
Local $sWinHttpVer = ^ ERROR

 

This function comes from the WinHTTP UDF, which is a requirement of the Webdriver UDF. It needs to be downloaded and installed into the appropriate folder on your system so that it can be found by Scite.

[SaeidN]

@Danp2 I made both v105. Looks like my winhttp.au3 was not updated. I updated manually from the link in WD_core. Now I'm getting this in full mode.

expandcollapse popup

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" /ErrorStdOut "C:\Users\me\Documents\wd_demo.au3"    
_WD_Option ==> Success [0] : Parameters:   Option=errormsgbox   Value=False
_WD_Option ==> Success [0] : Parameters:   Option=OutputDebug   Value=False
_WD_GetBrowserPath ==> Success [0 / 0] : Parameters:    Browser=Chrome
_WD_GetBrowserVersion ==> Success [0 / 0] : Parameters:    Browser=Chrome
_WD_GetWebDriverVersion ==> Success [0 / 0] : Parameters:    Dir=C:\Users\me\Documents\    EXE=chromedriver.exe
__WD_GetLatestWebdriverInfo ==> Success [0 / 0]
_WD_UpdateDriver ==> Success [0 / 0] : DriverCurrent = 105.0.5195.52 : DriverLatest = 105.0.5195.52
> UpdateResult = False
_WD_Option ==> Success [0] : Parameters:   Option=Driver   Value=chromedriver.exe
_WD_Option ==> Success [0] : Parameters:   Option=Port   Value=9515
_WD_Option ==> Success [0] : Parameters:   Option=DriverParams   Value=--verbose --log-path="C:\Users\me\Documents\chrome.log"
_WD_CapabilitiesStartup: #148 : > {"capabilities":"{}"} > IsObj = 1
_WD_CapabilitiesAdd: #177 :      $key = alwaysMatch     $value1 = chrome     $value2 = 
_WD_CapabilitiesAdd: #205 :  $_WD_NOTATION__MATCHTYPE = [capabilities][alwaysMatch] $_WD_NOTATION__SPECIFICVENDOR = ["goog:chromeOptions"]
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd: #177 :      $key = w3c     $value1 = True     $value2 = 
_WD_CapabilitiesAdd: #257 #273 :      $key = w3c     $value1 = True     $value2 =     $s_Notation = [capabilities][alwaysMatch]["goog:chromeOptions"]["w3c"]   <<<<  True
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd: #177 :      $key = excludeSwitches     $value1 = enable-automation     $value2 = 
_WD_CapabilitiesAdd: #239 #273 :      $key = excludeSwitches     $value1 = enable-automation     $value2 =     $s_Notation = [capabilities][alwaysMatch]["goog:chromeOptions"]["excludeSwitches"][0]   <<<<  enable-automation
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesDump: JSON structure starts below: 986
{
    "capabilities":{
        "alwaysMatch":{
            "goog:chromeOptions":{
                "w3c":true,
                "excludeSwitches":[
                    "enable-automation"
                ]
            }
        }
    }
}
_WD_CapabilitiesDump: JSON structure ends above.
> wd_demo.au3: _WD_Startup
_WD_IsLatestRelease ==> Success [0] : True
_WD_Startup: OS:    WIN_11 WIN32_NT 22000 
_WD_Startup: AutoIt:    3.3.16.0
_WD_Startup: Webdriver UDF: 0.10.1 (Up to date)
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --verbose --log-path="C:\Users\me\Documents\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --verbose --log-path="C:\Users\me\Documents\chrome.log" 
_WD_Startup ==> Success [0]
> wd_demo.au3: _WD_CreateSession
"C:\Users\me\Documents\WinHttp.au3" (932) : ==> Variable used without being declared.:
Case $WINHTTP_OPTION_AUTOLOGON_POLICY, $WINHTTP_OPTION_CODEPAGE, $WINHTTP_OPTION_CONFIGURE_PASSPORT_AUTH, $WINHTTP_OPTION_CONNECT_RETRIES, $WINHTTP_OPTION_CONNECT_TIMEOUT, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_OPTION_ENABLE_FEATURE, $WINHTTP_OPTION_ENABLETRACING, $WINHTTP_OPTION_MAX_CONNS_PER_1_0_SERVER, $WINHTTP_OPTION_MAX_CONNS_PER_SERVER, $WINHTTP_OPTION_MAX_HTTP_AUTOMATIC_REDIRECTS, $WINHTTP_OPTION_MAX_HTTP_STATUS_CONTINUE, $WINHTTP_OPTION_MAX_RESPONSE_DRAIN_SIZE, $WINHTTP_OPTION_MAX_RESPONSE_HEADER_SIZE, $WINHTTP_OPTION_READ_BUFFER_SIZE, $WINHTTP_OPTION_RECEIVE_TIMEOUT, $WINHTTP_OPTION_RECEIVE_RESPONSE_TIMEOUT, $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REJECT_USERPWD_IN_URL, $WINHTTP_OPTION_REQUEST_PRIORITY, $WINHTTP_OPTION_RESOLVE_TIMEOUT, $WINHTTP_OPTION_SECURE_PROTOCOLS, $WINHTTP_OPTION_SECURITY_FLAGS, $WINHTTP_OPTION_SECURITY_KEY_BITNESS, $WINHTTP_OPTION_SEND_TIMEOUT, $WINHTTP_OPTION_SPN, $WINHTTP_OPTION_USE_GLOBAL_SERVER_CREDENTIALS, $WINHTTP_OPTION_WORKER_THREAD_COUNT, $WINHTTP_OPTION_WRITE_BUFFER_SIZE, $WINHTTP_OPTION_DECOMPRESSION, $WINHTTP_OPTION_UNSAFE_HEADER_PARSING
Case $WINHTTP_OPTION_AUTOLOGON_POLICY, $WINHTTP_OPTION_CODEPAGE, $WINHTTP_OPTION_CONFIGURE_PASSPORT_AUTH, $WINHTTP_OPTION_CONNECT_RETRIES, $WINHTTP_OPTION_CONNECT_TIMEOUT, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_OPTION_ENABLE_FEATURE, $WINHTTP_OPTION_ENABLETRACING, $WINHTTP_OPTION_MAX_CONNS_PER_1_0_SERVER, $WINHTTP_OPTION_MAX_CONNS_PER_SERVER, $WINHTTP_OPTION_MAX_HTTP_AUTOMATIC_REDIRECTS, $WINHTTP_OPTION_MAX_HTTP_STATUS_CONTINUE, $WINHTTP_OPTION_MAX_RESPONSE_DRAIN_SIZE, $WINHTTP_OPTION_MAX_RESPONSE_HEADER_SIZE, $WINHTTP_OPTION_READ_BUFFER_SIZE, $WINHTTP_OPTION_RECEIVE_TIMEOUT, $WINHTTP_OPTION_RECEIVE_RESPONSE_TIMEOUT, $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REJECT_USERPWD_IN_URL, $WINHTTP_OPTION_REQUEST_PRIORITY, $WINHTTP_OPTION_RESOLVE_TIMEOUT, $WINHTTP_OPTION_SECURE_PROTOCOLS, $WINHTTP_OPTION_SECURITY_FLAGS, $WINHTTP_OPTION_SECURITY_KEY_BITNESS, $WINHTTP_OPTION_SEND_TIMEOUT, $WINHTTP_OPTION_SPN, $WINHTTP_OPTION_USE_GLOBAL_SERVER_CREDENTIALS, $WINHTTP_OPTION_WORKER_THREAD_COUNT, $WINHTTP_OPTION_WRITE_BUFFER_SIZE, ^ ERROR
>Exit code: 1

 

[Danp2]

@SaeidNThose constants are defined in WinHttpConstants.au3, which is part of the WinHTTP UDF. Did you place this file where it can be found? 🤨

[SaeidN]

@Danp2 Looks llike winhttpconstant file wasn't updated. I have a version for 10 years ago from autoit github. I've downloaded the new one from here. And it worked.

To open IE only 1 line is needed. but to open chrome looks like at least 6-7 lines is needed plus 6-7 au3 files, which takes long time to look at it and update my IE code to chrome version.

Thanks anyway.

[Danp2]

@SaeidNYes... Webdriver is definitely more complicated. @mLipok has a section on the Webdriver wiki page dealing with Translate IE UDF to WebDriver.

[mLipok]

16 minutes ago, Danp2 said:

Webdriver is definitely more complicated.

It's kind of a habit. The user should start using WebDriver to get into the habit of using it.
It just takes time.

24 minutes ago, SaeidN said:

To open IE only 1 line is needed. but to open chrome looks like at least 6-7 lines is needed plus 6-7 au3 files

Something at the expense of something.
Greater flexibility of the solution, more possibilities, at the expense of the amount of additional code.

1 hour ago, SaeidN said:

It's been one hour I'm looking everywhere to see how I can open a url (let's say www.yahoo.com) in chrome using webdriver UDF, and didn't find anything.

did you look here

https://www.autoitscript.com/wiki/WebDriver

and here

 

??