Webdriver, Websockets, and Chrome DevTools Protocol ... Oh my! ​😜

[Danp2]

Hi All,

I have been working on an enhancement to the WebDriver UDF that will make _WD_ExecuteCDPCommand compatible with all major browsers instead of Chrome only. This involves using WebSockets to pass commands back and forth to the browser.

Attached to this post are two files --

• winhttp_websocket.au3 -- This is a modified version of code originally posted to the forum by @FireFox.
• genericcdp.au3 -- This contains the enhanced version of _WD_ExecuteCDPCommand (original version by @TheDcoder) and a new function named _WD_GetCDPSettings

You can run genericcdp.au3 for a brief example of how the new functionality works. Comment / uncomment the appropriate lines near the beginning to control while browser gets tested.

Prerequisites

• You'll obviously need the WebDriver UDF installed and working for this code to have a chance of working correct. 🙄
• You need to be running an OS that supports WebSockets natively (ie: Windows 8 / Windows Server 2012 or newer)

Caveats
Chromium-based browsers return some unexpected status codes in response to _WinHttpWebSocketClose and _WinHttpWebSocketQueryCloseStatus, so I've included checks to ignore these.

Needs
I would appreciate it if I could get some feedback / suggestions for improvement on these proposed changes.

FWIW, I have only performed these basic CDP commands. Feel free to experiment with other CDP commands and report back on your successes / failures.

Thanks,

Dan

genericcdp.au3 winhttp_websocket.au3

Edited July 21, 2021 by Danp2
Added OS restrictions

[TheXman]

You are using websocket APIs that state that they require a minimum client of Windows 8 or a minimum server of Windows 2012.  Shouldn't that be listed as a prerequisite? 

I just ran the genericCDP script using the latest 64-bit Firefox on Windows 7 and I got an @error = 2 from the _WD_ExecuteCDPCommand() function.  I ran the same test on a Windows 10 client and it worked as expected.

The log from Windows 7 PC is below:

_WD_Startup: OS:    WIN_7 WIN32_NT 7601 Service Pack 1
_WD_Startup: AutoIt:    3.3.14.5
_WD_Startup: WD.au3:    0.4.0.5
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    C:\Portable Apps\AutoIt3\Include\MyIncludes\WebDriver\Drivers\geckodriver_x64.exe
_WD_Startup: Params:    --log trace
_WD_Startup: Port:  4444
__WD_Post: URL=HTTP://127.0.0.1:4444/session; $sData={"capabilities":{"alwaysMatch": {"moz:debuggerAddress": true, "moz:firefoxOptions": {"log": {"level": "trace"}}}}}
__WD_Post: StatusCode=200; ResponseText={"value":{"sessionId":"38491b67-3b02-4638-a7fd-077cf118e04f","capabilities":{"acceptInsecureCerts":f...
_WD_CreateSession: {"value":{"sessionId":"38491b67-3b02-4638-a7fd-077cf118e04f","capabilities":{"acceptInsecureCerts":false,"browserName":"firefox","browserVersion":"90.0","moz:accessibilityChecks":false,"moz:buildID":"20210705185941","moz:debuggerAddress":"localhost:60972","moz:geckodriverVersion":"0.29.1","moz:headless":false,"moz:processID":6528,"moz:profile":"C:\\Temp\\rust_mozprofile0i0kl3","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"windows","platformVersion":"6.1","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}}
__WD_Post: URL=HTTP://127.0.0.1:4444/session/38491b67-3b02-4638-a7fd-077cf118e04f/url; $sData={"url":"http://google.com"}
__WD_Post: StatusCode=200; ResponseText={"value":null}...
_WD_Navigate: {"value":null}
__WD_Post: URL=HTTP://127.0.0.1:4444/session/38491b67-3b02-4638-a7fd-077cf118e04f/window/new; $sData={"type":"tab"}
__WD_Post: StatusCode=200; ResponseText={"value":{"handle":"64c6d394-1b1b-46d6-bb8f-d1599fbfe0f2","type":"tab"}}...
_WD_Window: {"value":{"handle":"64c6d394-1b1b-46d6-bb8f-d1599fbfe0f2","type":"tab"}}...
__WD_Post: URL=HTTP://127.0.0.1:4444/session/38491b67-3b02-4638-a7fd-077cf118e04f/window; $sData={"handle":"64c6d394-1b1b-46d6-bb8f-d1599fbfe0f2"}
__WD_Post: StatusCode=200; ResponseText={"value":null}...
_WD_Window: {"value":null}...
__WD_Post: URL=HTTP://127.0.0.1:4444/session/38491b67-3b02-4638-a7fd-077cf118e04f/url; $sData={"url":"http://yahoo.com"}
__WD_Post: StatusCode=200; ResponseText={"value":null}...
_WD_Navigate: {"value":null}
__WD_Get: URL=http://localhost:60972/json/list
__WD_Get: StatusCode=200; $iResult = 0; $sResponseText=[
    {
        "description": "",
        "devtoolsFrontendUrl": null,
        "faviconUrl": "",
        "id": "a881de60-63d7-...
_WD_ExecuteCDPCommand: ...
_WD_ExecuteCDPCommand ==> Socket Error: SetOption error

 

Edited July 17, 2021 by TheXman

[Danp2]

55 minutes ago, TheXman said:

You are using websocket APIs that state that they require a minimum client of Windows 8 or a minimum server of Windows 2012.  Shouldn't that be listed as a prerequisite? 

Very true. I will update the OP.

55 minutes ago, TheXman said:

I just ran the genericCDP script using the latest 64-bit Firefox on Windows 7 and I got an @error = 2

That makes sense given the above prerequisite.

P.S. Thanks for testing!

Edited July 17, 2021 by Danp2

[Danp2]

Anyone want to suggest changes or improvements?

Also would appreciate input on why the WS functions return an error when closing a websocket with Chrome & MSEdge. Is this just the way it is or am I doing somehting wrong? 🤔

[TheXman]

I only use Firefox and my primary PCs are Windows 7.  However, I do have a Windows 10 PC with MS Edge that I can run tests on.  I'll take a look at the error when you're closing a websocket, on MS Edge, and see if I can find the root cause.  Can you explain the issue that you are seeing so I know that I'm chasing the right problem? If you have implemented any workarounds that may be masking the issue, can you provide either a script that reproduces the issue or tell me what steps I need to take to reproduce it?

Edited July 19, 2021 by TheXman

[mLipok]

Vacation Mode.

Sorry 

 

[Danp2]

@TheXmanThat's for the offer. Look at the error checking in _WD_ExecuteCDPCommand following the calls to _WinHttpWebSocketClose and _WinHttpWebSocketQueryCloseStatus. I don't encounter these error codes (ERROR_WINHTTP_CONNECTION_ERROR and ERROR_INVALID_OPERATION) with Firefox, but I do consistently with Chrome and Edge.

@mLipokI knew it. Otherwise, you would have already torn it apart and complete rewritten it. 😜

[TheXman]

I just starting taking a look.  Before getting too deep, I noticed something that I couldn't figure out.

 

Microsoft defines the _WINHTTP_WEB_SOCKET_CLOSE_STATUS enumerators as: (starting at 0)

typedef enum _WINHTTP_WEB_SOCKET_CLOSE_STATUS {
  WINHTTP_WEB_SOCKET_SUCCESS_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_ENDPOINT_TERMINATED_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_PROTOCOL_ERROR_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_INVALID_DATA_TYPE_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_EMPTY_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_ABORTED_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_INVALID_PAYLOAD_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_POLICY_VIOLATION_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_MESSAGE_TOO_BIG_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_UNSUPPORTED_EXTENSIONS_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_SERVER_ERROR_CLOSE_STATUS,
  WINHTTP_WEB_SOCKET_SECURE_HANDSHAKE_ERROR_CLOSE_STATUS
} WINHTTP_WEB_SOCKET_CLOSE_STATUS;

However, in the winhttp_websocket.au3 UDF lib, the enum is defined as: (starting at 1000)

Global Enum _
        $WINHTTP_WEB_SOCKET_SUCCESS_CLOSE_STATUS = 1000, _
        $WINHTTP_WEB_SOCKET_ENDPOINT_TERMINATED_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_PROTOCOL_ERROR_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_INVALID_DATA_TYPE_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_EMPTY_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_ABORTED_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_INVALID_PAYLOAD_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_POLICY_VIOLATION_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_MESSAGE_TOO_BIG_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_UNSUPPORTED_EXTENSIONS_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_SERVER_ERROR_CLOSE_STATUS, _
        $WINHTTP_WEB_SOCKET_SECURE_HANDSHAKE_ERROR_CLOSE_STATUS

I haven't been able to figure out why you start your enumerators at 1000?  That would throw off all of the statuses by 1000, including a successful status. 

Edited July 19, 2021 by TheXman
Added link to MS enum definition

[Danp2]

AFAICS, 1000 is the correct starting value. If you look at the original code here, @FireFoxuses the following definition --

Global Const $WINHTTP_WEB_SOCKET_SUCCESS_CLOSE_STATUS = 1000

I just changed / expanded it to an enum. This also seems to match other definitions that I found online.

[TheXman]

That is very odd.  Looking at sources other than Microsoft, I do see references to the enumerator starting at 1000.

I added a link, in my previous post, to the definition on Microsoft's site.  I'll keep digging to see if I can find more information as to why there's a discrepancy.

[TheXman]

After reading thru the WebSocket RFC as it relates to closing a WebSocket connection, in this specific use case where you are just communicating with the browser (CDP), there's really no need for the client to close the WebSocket, it'll be closed when the underlying TCP connection is closed.  I guess if you wanted to do some tidying up, you could close the WebSocket's handle before exiting the script.    I commented out all of the relevant "close" logic (WebSocketShutdown, WebSocketClose, & WebSocketQueryCloseStatus) and saw no ill effects or remnants left behind.

There's definitely something different between the way Firefox handles the CDP commands and the way MS Edge handles them.  I noticed when running the script in MS Edge, the script switches from the Yahoo page to the Google page and then back to the Yahoo page before exiting.  When running the script against Firefox, it doesn't do the last switch from Google back to Yahoo before exiting, it exits while on the Google page.

Edited July 20, 2021 by TheXman

[Danp2]

Pretty sure what you are encountering is a different in the order that items appear in output from /json/list. You would experience the opposite if you change the line

For $x = UBound($aList, $UBOUND_COLUMNS) - 1 to 1 Step -1

to

For $x = 1 to UBound($aList, $UBOUND_COLUMNS) - 1

It was just a Q&D example, so feel free to suggest improvements.

Appreciate the feedback on the WebSocket handling. I think that I'll leave the coding as-is for now.

[seadoggie01]

Does this mean I might be able to manipulate discarded tabs now? Time to dive in!

[Danp2]

@seadoggie01What do you mean? Something like reopening a previously closed tab?

[seadoggie01]

@Danp2 I'm not sure how to use this UDF yet

I was hoping to find something to help keep tabs from being discarded (same thing as here). I've still been poking around the issue, but for now I keep the browser visible on my 3rd screen.

[Danp2]

@seadoggie01OIC. I forgot about this earlier discussion.

[jugador]

@Danp2 Pre-defined status codes when sending a Close frame

editing the post with the link  https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent

Edited July 27, 2021 by jugador

[Danp2]

@jugador You may want to update your post with a link to the source.

[junkew]

I found another topic on this CDP protocol using pipes Automate Chrome / Edge using VBA - CodeProject
 

Quote

The low-level access to the CDP protocol is avaible by two means: Either Edge starts a small Webserver on a specific port or via pipes. The Webserver lacks any security features. Any user on the computer has access to the webserver. This may pose no risks on single user computers or dedicated virtual containers. But if the process is run on a terminal server with more than one user, this is not acceptable. That's why the code uses pipes to communicate with Edge.

 

[FrancoBordin]

On 7/17/2021 at 7:21 PM, Danp2 said:

I would appreciate it if I could get some feedback / suggestions for improvement on these proposed changes.

Hi there.

I don't exactly know whether this post of mine should be classified as a "suggestion" or "help-request" 😀 Maybe s'thing in between.

I hoped i could use " _WD_ExecuteCDPCommand" to call the cdp command "Network.Headers" in order to retrieve the request/response headers of the page/tab currently opened in (and controlled by) a webdriver session (similar to what i usually do for cookies with "_WD_Cookies"). Unfortunately i saw that "Network.Headers" is listed as a "Type" (not as a "method") in the official cdp documentation linked in your post, so i'm currently unable to translate it into actual autoit code in order to perform a test.

Here's my sample code

Local $sCapabilities = ""
If FileExists("geckodriver.exe") Then
    _WD_UpdateDriver('Firefox')
    $sCapabilities = Call(SetupGecko, False)
ElseIf FileExists("chromedriver.exe") Then
    _WD_UpdateDriver('Chrome')
    $sCapabilities = Call(SetupChrome, False)
ElseIf FileExists("msedgedriver.exe") Then
    _WD_UpdateDriver('MSEdge')
    $sCapabilities = Call(SetupEdge, False)
ElseIf FileExists("operadriver.exe") Then
    _WD_UpdateDriver('Opera')
    $sCapabilities = Call(SetupOpera, False)
EndIf

If not StringIsSpace($sCapabilities) Then
   Local $iWebDriver_PID = _WD_Startup()
   Local $sSession = _WD_CreateSession($sCapabilities)
   _WD_Window($sSession, "Maximize")
   _WD_Navigate($sSession, "http://www.somesite.com")
   If MsgBox($MB_OKCANCEL + $MB_SYSTEMMODAL + $MB_TOPMOST, "Awaiting confirmation", "Please click <OK> as soon as navigation is completed") = $idOk Then

      ;retrieving session cookies
      Local $cCookiesJson = _WD_Cookies($sSession, "GETALL")

      ;retrieving session last response headers (currently i'm unable to do this)
      ?????????

   Endif
   _WD_DeleteSession($sSession)
   _WD_Shutdown($iWebDriver_PID)
Endif

Is that already possible?

If not so, here's my implicit suggestion 😀

Ciao

Franco

P.S.: obviously it would not be a big deal even if should be possibile only on chrome