WinHTTP - Setting option fails

[BBs19]

Hi guys,

I am trying to automate searching on a website at work and I am failing so far with the very basic step of HTTP Get  The website is only reachable via https.

I usually work with "WinHttp.WinHttpRequest.5.1" and had no problems automating anything so far, but this website is a tricky one, it seems like it wants some kind of client certificate, or at least it wants it but doesn't require it. I removed all the client certificates in Firefox but the website still works, so I assume the client certificate is not necessary but optionally.

"WinHttp.WinHttpRequest.5.1" always throws an client certificate error when i try to send a HTTP Get request. A window opens and is asking for MyCard authentication.

Now I am trying to automate it using the WinHTTP UDF, because I found a promising option on the msdn page: https://msdn.microsoft.com/en-us/library/windows/desktop/aa384066(v=vs.85).aspx#WINHTTP_OPTION_CLIENT_CERT_CONTEXT

Quote

If the server requests the certificate but does not require it, the application can specify this option to indicate that it does not have a certificate. The server can choose another authentication scheme or allow anonymous access to the server. The application provides theWINHTTP_NO_CLIENT_CERT_CONTEXT macro in the lpBuffer parameter of WinHttpSetOption as shown in the following code example.

 

BOOL fRet = WinHttpSetOption ( hRequest,
                               WINHTTP_OPTION_CLIENT_CERT_CONTEXT,
                               WINHTTP_NO_CLIENT_CERT_CONTEXT,
                               0); 

Now my problem is that i just can't call this option with the WinHTTP UDF. It always returns @error 3( IsPtr returned false).

I am really bad with dll calls, and I couldn't find any example for this option. I am sure the answer is pretty easy...

Does anyone know how to properly call the above option?

 

here an example:

expandcollapse popup

#AutoIt3Wrapper_AU3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#include "WinHttp.au3"

Global $hHost = "msdn.microsoft.com"
Global $hGet = "/en-us/library/windows/desktop/aa384066(v=vs.85).aspx"

; Initialize
Global $hOpen = _WinHttpOpen("")

If @error Then
    MsgBox(48, "Error", "Error initializing the usage of WinHTTP functions.")
    Exit 1
EndIf

; Specify what to connect to
Global $hConnect = _WinHttpConnect($hOpen, $hHost, $INTERNET_DEFAULT_HTTPS_PORT) ; <- yours here
If @error Then
    MsgBox(48, "Error", "Error specifying the initial target server of an HTTP request.")
    _WinHttpCloseHandle($hOpen)
    Exit 2
EndIf

; Create request
Global $hRequest = _WinHttpOpenRequest($hConnect, "GET", $hGet, "HTTP/1.1", $WINHTTP_NO_REFERER, $WINHTTP_DEFAULT_ACCEPT_TYPES, $WINHTTP_FLAG_SECURE)
If @error Then
    MsgBox(48, "Error", "Error creating an HTTP request handle.")
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)
    Exit 3
EndIf

_WinHttpSetOption($hRequest, $WINHTTP_OPTION_SECURITY_FLAGS, BitOR($SECURITY_FLAG_IGNORE_UNKNOWN_CA, $SECURITY_FLAG_IGNORE_CERT_DATE_INVALID, $SECURITY_FLAG_IGNORE_CERT_CN_INVALID, $SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE))

;#######################################################################################################
Global Const $WINHTTP_NO_CLIENT_CERT_CONTEXT = Null
_WinHttpSetOption($hRequest, $WINHTTP_OPTION_CLIENT_CERT_CONTEXT, $WINHTTP_NO_CLIENT_CERT_CONTEXT);<<<<<<<<<<<<<<<<<

MsgBox(16, "", @error)
Exit
;#######################################################################################################

; Send it
_WinHttpSendRequest($hRequest)
If @error Then
    MsgBox(48, "Error", "Error sending specified request.")
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)
    Exit 4
EndIf

; Wait for the response
_WinHttpReceiveResponse($hRequest)
If @error Then
    MsgBox(48, "Error", "Error waiting for the response from the server.")
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)
    Exit 5
EndIf

; See if there is data to read
Global $sChunk, $sData
If _WinHttpQueryDataAvailable($hRequest) Then
    ; Read
    While 1
        $sChunk = _WinHttpReadData($hRequest)
        If @error Then ExitLoop
        $sData &= $sChunk
    WEnd
    MsgBox(0, "", $sData)
Else
    MsgBox(48, "Error", "Site is experiencing problems.")
EndIf

; Close handles when they are not needed any more
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

 

 

EDIT: Ok it was very easy and it works now, just needed to execute the following dllcall on the request:

DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpSetOption", "handle", $hRequest, "dword", $WINHTTP_OPTION_CLIENT_CERT_CONTEXT, "ptr", NULL, "dword", 0)

Edited May 30, 2016 by BBs19
added solution

[AutoBert]

I never worked with the http.au3 but found this small example: _WinHttpSimpleSendSSLRequest.au3

[BBs19]

Thanks, but I don't think that you understood what I wrote

It is not about a simple https request but a problem with the server which requests a client certificate but doesn't require it. I need a way to set the mentioned option of WinHTTP above to tell the server that I don't have a client certificate.

[AutoBert]

Yes, i don't understand:

Global $hConnect = _WinHttpConnect($hOpen, $hHost, $INTERNET_DEFAULT_HTTPS_PORT)

why you are using HTTPS-Port on an non HTTPS server.

[orbs]

16 hours ago, BBs19 said:

it wants it but doesn't require it. I removed all the client certificates in Firefox but the website still works, so I assume the client certificate is not necessary but optionally.

you are confusing "client certificate" with "trusted certificate authority". you can strip a browser from whatever client certificate it may have, but it still trusts a list of certificate authorities (abbr. CA). that list is updated regularly (not too often though) by the browser vendor, and is used to determine if any HTTPS site is trustworthy.

i once encountered this issue but i did not investigate it to full resolution as i found a practical workaround - i enlisted the curl utility, which is distributed with its own trusted CA list. so i cannot offer any more assistance other then pinpointing the difference - what you need is to configure your HTTPS to utilize the trusted CA list (which you can export from your browser). if you are dealing with an intranet site, i tend to believe the easiest solution would be to first configure your browser to trust the local CA, then export that.

[BBs19]

5 hours ago, orbs said:

you are confusing "client certificate" with "trusted certificate authority". you can strip a browser from whatever client certificate it may have, but it still trusts a list of certificate authorities (abbr. CA). that list is updated regularly (not too often though) by the browser vendor, and is used to determine if any HTTPS site is trustworthy.

i once encountered this issue but i did not investigate it to full resolution as i found a practical workaround - i enlisted the curl utility, which is distributed with its own trusted CA list. so i cannot offer any more assistance other then pinpointing the difference - what you need is to configure your HTTPS to utilize the trusted CA list (which you can export from your browser). if you are dealing with an intranet site, i tend to believe the easiest solution would be to first configure your browser to trust the local CA, then export that.

But isn't that to authenticate the website certificate ? I have already added the option to ignore ssl errors above, so I am not sure how this would be relevant.

I also do get a "client authentication failed" with both WinHTTP and also when using "WinHttp.WinHttpRequest.5.1". I don't even get to send the request.

I need a way to authenticate the user to the server and not other other way around. The MSDN site explains it well. Maybe I am confusing something completely, but I think if I was able to set the option suggested on the MSDN site, I would see

 

@AutoBert

Please read the first post, that script is just an example, I can't post the real site as it is an intranet site.

Edited May 27, 2016 by BBs19

[BBs19]

In case anyone ever needs this, it was pretty easy, just needed to execute following dllcall for the request handle:

 

DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpSetOption", "handle", $hRequest, "dword", $WINHTTP_OPTION_CLIENT_CERT_CONTEXT, "ptr", NULL, "dword", 0)

This way we tell the server that we do not have any client certificate. It works now, the server replies properly.