Retrieve file name using WinHTTP

[Scagnetti]

I need to be able to retrieve a file name from a dynamic link so I tried the >following code by weaponx:

$oHTTP = ObjCreate('winhttp.winhttprequest.5.1')
$oHTTP.Open('POST', 'https://linkgoeshere.com/random', 1)
$oHTTP.SetRequestHeader('Content-Type','application/x-www-form-urlencoded')
;$oHTTP.setTimeouts(5000, 5000, 15000, 15000)
$oHTTP.Send()
$oHTTP.WaitForResponse
$ContentDisposition = $oHTTP.GetResponseHeader("Content-Disposition")
$array = StringRegExp($ContentDisposition, 'filename="(.*)"',3)
ConsoleWrite($array[0] & @CRLF)
ConsoleWrite($oHTTP.GetAllResponseHeaders())

*the links posted here have been changed as they are private in nature

When trying the above code I get:

The requested action with this object has failed.:
$ContentDisposition = $oHTTP.GetResponseHeader("Content-Disposition")
$ContentDisposition = $oHTTP.GetResponseHeader("Content-Disposition")^ ERROR

weaponx's original code works fine for me, so I'm assuming this error is probably because 1) the link uses SSL (https) or 2) because the website has a self-signed certificate.

As a last resort, I was able to get the functionality I wanted using wget:

wget --no-check-certificate --content-disposition -N "https://linkgoeshere.com/random" -P %SystemDrive%test 

and I verified that it does preserve the original file name, but I'd prefer to figure out how to do this in autoit w/o an external program if possible.

[edit - changed topic title + minor edits]

Edited July 23, 2013 by Scagnetti

[Scagnetti]

After looking into this further, I looked over WinHttpRequestOption enumeration and did some inspection with wireshark while using wget.  When I remove the --no-check-certificate flag from wget, the download will not go through, mentioning "self-signed certificate encountered"

I tried the following:

$oHTTP = ObjCreate('winhttp.winhttprequest.5.1')
$oHTTP.Option(4) = 0x3300 ;WinHttpRequestOption_SslErrorIgnoreFlags
$oHTTP.Option(9) = 0x0080 ;WinHttpRequestOption_SecureProtocols
$oHTTP.Open('POST', 'https://linkgoeshere.com/random', 1)
$oHTTP.SetRequestHeader('Content-Type','application/x-www-form-urlencoded')
$oHTTP.Send()
$oHTTP.WaitForResponse
ConsoleWrite($oHTTP.GetAllResponseHeaders())

and got the following output from the console:

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: ns_s=c9d0bc2acf55c0eb862b45ad1050a2cc4a697e22; path=/; secure; HttpOnly
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8

What I'm looking for here is Content-Disposition, which is not present - any suggestions?  It looks like WinHttpRequestOption_EnableRedirects=VARIANT_TRUE by default, so I shouldn't need to worry about that.

[trancexx]

I understand that working with COM objects is very much simpler than working with flat APIs but in this case I would suggest trying WinHttp.au3 UDF. Check for function called _WinHttpQueryHeaders(). You should query request header for $WINHTTP_QUERY_CONTENT_DISPOSITION.

UDF can be found in examples.

[Scagnetti]

I gave it a shot, but some of this is over my head

#include "WinHttp.au3"

Global $sHost = "https://linkgoeshere.com"
Global $sTarget = "name=Test&code=1234"
Global $hHttpOpen = _WinHttpOpen()
Global $hConnect = _WinHttpConnect($hHttpOpen, $sHost)
Global $hRequest = _WinHttpOpenRequest($hConnect)
Global $hHeader = _WinHttpQueryHeaders($hRequest,$WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($hHeader)

I'm getting a empty string in the console, which indicates a failure.

Edited July 23, 2013 by Scagnetti

[trancexx]

Based on the code you posted earlier it could be like this:

#include "WinHttp.au3"

Global $sHost = "linkgoeshere.com"
Global $sTarget = "name=Test&code=1234" ; if you say so (the code you posted earlier has this diffrent)

$hHttpOpen = _WinHttpOpen()
$hConnect = _WinHttpConnect($hHttpOpen, $sHost, $INTERNET_DEFAULT_HTTPS_PORT)
$hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "POST", $sTarget) ; "POST" is from your earlier code

$sDisp = _WinHttpQueryHeaders($hRequest,$WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($sDisp)

; ...Close handles here...

...That's written on mobile phone so excuse my French.

[Scagnetti]

Still getting an empty string so I must be doing something wrong.  I tried the link using "Live HTTP Headers" add-on in Firefox and got the following:

 

GET /random_dynamic_link HTTP/1.1
Host: linkgoeshere.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: ns_s=b0f0924eca01828a0f9c54441c047f967a483eb8
Connection: keep-alive

HTTP/1.1 200 OK
Date: Tue, 23 Jul 2013 18:54:17 GMT
Server: Server
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: invalidextension
Cache-Control: private, max-age=0, must-revalidate, post-check=0, pre-check=0
Content-Disposition: attachment; filename=randomized_name.exe
Set-Cookie: ns_s=4c50d43f189590e3022d99d5cb2df4acd1499f22; path=/; secure; HttpOnly
Content-Length: 1828544
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/x-executable

 

I did some minor edits to remove sensitive information (edits are in bold/red) - but hopefully this may help with knowing what is going on.  My main concern is getting the randomized name for the .exe since I can't preserve the original name using inetget.

[Scagnetti]

I realize that not posting a working link that can be tested makes it more difficult to figure out the problem, but I couldn't due to privacy concerns.

I was able to get permission to use an alternate link, that is more used for demo/testing purposes though, which has the same functionality: https://kboxdemo.bomgar.com/download_customer_connector.ns?issue_menu=1&id=1&customer_name=Name&download=1

#include "WinHttp.au3"

Global $sHost = "kboxdemo.bomgar.com"
Global $sTarget = "download_customer_connector.ns?issue_menu=1&id=1&customer_name=Name&download=1"

$hHttpOpen = _WinHttpOpen()
$hConnect = _WinHttpConnect($hHttpOpen, $sHost, $INTERNET_DEFAULT_HTTPS_PORT)
$hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "POST", $sTarget)

$sDisp = _WinHttpQueryHeaders($hRequest,$WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($sDisp)

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hHttpOpen)

Still getting an empty string, but hopefully that'll be more helpful - and thank you trancexx for your help.

[GMK]

This works for me:

#include "WinHttp.au3"

Global $sHost = "kboxdemo.bomgar.com"
Global $sTarget = "download_customer_connector.ns?issue_menu=1&id=1&customer_name=Name&download=1"

Global $hHttpOpen = _WinHttpOpen()
Global $hConnect = _WinHttpConnect($hHttpOpen, $sHost, $INTERNET_DEFAULT_HTTPS_PORT)
Global $hRequest = _WinHttpOpenRequest($hConnect, "GET", $sTarget, Default, Default, Default, $WINHTTP_FLAG_SECURE)
_WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)

Global $sDisp = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($sDisp & @CRLF)

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hHttpOpen)

Edited July 24, 2013 by GMK

[Scagnetti]

Thanks GMK, I had a feeling that User-Agent would be needed.  I tried your code but I'm still getting an empty string.  I'm using WinHttp v1.6.3.2 and SciTE 3.30.  anything I should be doing differently?

[GMK]

Sorry...try it again now (edited previous post). I started to change it, decided not to, but didn't undo some changes.

[Scagnetti]

lol, I made the exact same changes and verified that it did work but you beat me.  thank you so much GMK and thank you again trancexx for WinHttp.au3

Edited July 24, 2013 by Scagnetti

[GMK]

You're welcome!

Here's what I was going to do, but decided against earlier:

#include "WinHttp.au3"

Global $sURL = "https://kboxdemo.bomgar.com/download_customer_connector.ns?issue_menu=1&id=1&customer_name=Name&download=1"
Global $aCracked = _WinHttpCrackUrl($sURL)

Global $hHttpOpen = _WinHttpOpen()
Global $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
Global $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
_WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)

Global $sDisp = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($sDisp & @CRLF)

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hHttpOpen)