WinHTTP functions

[DUNGYEUANH]

How to modify cookie values ​​are handled by WinHTTP ?

[trancexx]

It depends. What exactly you want to do?

[wraithdu]

I wanted to pass on some information I came across while playing with async requests. First off, they just don't work. I can't successfully complete one async call unless the read buffer is large enough and all the data is retrieved in one call to ReadData. Crashes abound.

That said, there are two errors in the UDF in regard to async. In both the QueryDataAvailable and ReadData functions, the lpdwNumberOfBytesRead parameter MUST be NULL, or MSDN warns risk of a memory fault. The data sizes should be retrieved in the callback function.

[trancexx]

Yes, I should probably remove async function.

The thing is that no one ever reported help file example not working. Probably nobody uses async function from the UDF. There were some not very successful attempts to do it by means of using custom dlls.

AutoIt's callback needs more work to get stuff like this working correctly and safely.

Edited March 21, 2013 by trancexx

[DUNGYEUANH]

It depends. What exactly you want to do?

When I access the web page, WinHTTP automatically save cookie values ​​(Example: "Cookie: PHPSESSID = 123456"), but I want to edit it (Example: "Cookie: PHPSESSID = 123456; abc = 123").So how do I do ?

Sorry, my english is not good

[trancexx]

When I access the web page, WinHTTP automatically save cookie values ​​(Example: "Cookie: PHPSESSID = 123456"), but I want to edit it (Example: "Cookie: PHPSESSID = 123456; abc = 123").So how do I do ?

Sorry, my english is not good

So first you read cookie, then you edit cookie string, then you disable automatic cookie handling for request handle, then you clear all cookies, then you manually set edited cookies, then you enable automatic cookie handling, then you send request.

[DUNGYEUANH]

How to clear all cookies ???

_WinHttpAddRequestHeaders( hRequest,"Cookie:", $WINHTTP_ADDREQ_FLAG_REPLACE)

It failed

[trancexx]

Automatic cookie handling should be disabled first.

_WinHttpSetOption($hRequest, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_DISABLE_COOKIES)

Edited March 23, 2013 by trancexx

[DUNGYEUANH]

@trancexx: thanks for your help, but I still can not get it to work, here is my test code

expandcollapse popup

#include "WinHttp.au3"

Opt("MustDeclareVars", 1)

Global $hOpen = _WinHttpOpen()
_WinHttpSetOption($hOpen, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_DISABLE_COOKIES)

Global $hConnect = _WinHttpConnect($hOpen, "vn.yahoo.com")
Global $hRequest = _WinHttpOpenRequest($hConnect, Default, "/?p=us")
_WinHttpAddRequestHeaders($hRequest, "Host: vn.yahoo.com")
_WinHttpAddRequestHeaders($hRequest, "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
_WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22")
_WinHttpAddRequestHeaders($hRequest, "Accept-Language: vi-VN,vi;q=0.8,fr-FR;q=0.6,fr;q=0.4,en-US;q=0.2,en;q=0.2")
_WinHttpAddRequestHeaders($hRequest, "Accept-Charset: windows-1258,utf-8;q=0.7,*;q=0.3")

_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)
Global $sChunk, $sData
If _WinHttpQueryDataAvailable($hRequest) Then
; Read
While 1
$sChunk = _WinHttpReadData($hRequest)
If @error Then ExitLoop
$sData &= $sChunk
WEnd
Else
MsgBox(48, "Error", "Site is experiencing problems.")
EndIf


$hRequest = _WinHttpOpenRequest($hConnect, Default, "/?p=us")
_WinHttpAddRequestHeaders($hRequest, "Host: vn.yahoo.com")
_WinHttpAddRequestHeaders($hRequest, "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
_WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22")
_WinHttpAddRequestHeaders($hRequest, "Accept-Language: vi-VN,vi;q=0.8,fr-FR;q=0.6,fr;q=0.4,en-US;q=0.2,en;q=0.2")
_WinHttpAddRequestHeaders($hRequest, "Accept-Charset: windows-1258,utf-8;q=0.7,*;q=0.3")
_WinHttpAddRequestHeaders($hRequest,"Cookie: 1234", $WINHTTP_ADDREQ_FLAG_REPLACE)
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)
Global $sChunk, $sData
If _WinHttpQueryDataAvailable($hRequest) Then
; Read
While 1
$sChunk = _WinHttpReadData($hRequest)
If @error Then ExitLoop
$sData &= $sChunk
WEnd
Else
MsgBox(48, "Error", "Site is experiencing problems.")
EndIf

[trancexx]

Disable cookies for the request, not for the session.

[DUNGYEUANH]

Can you give me an example ? Many thanks !!!

[wakillon]

DUNGYEUANH,

as stated by trancexx

use _WinHttpSetOption($hRequest, $WINHTTP_OPTION_DISABLE_FEATURE, $WINHTTP_DISABLE_COOKIES)

[trancexx]

I wanted to pass on some information I came across while playing with async requests. First off, they just don't work. I can't successfully complete one async call unless the read buffer is large enough and all the data is retrieved in one call to ReadData. Crashes abound.

That said, there are two errors in the UDF in regard to async. In both the QueryDataAvailable and ReadData functions, the lpdwNumberOfBytesRead parameter MUST be NULL, or MSDN warns risk of a memory fault. The data sizes should be retrieved in the callback function.

I figured out how to do correctly what you tried. I just had to tweak _WinHttpSimpleReadDataAsync example a bit. Even though original example works well (for me) it can be made correct even for pages with chunked transfer encoding and for buffers in sizes smaller than overall data size (for one read).

Those two errors for async flag for QueryDataAvailable and ReadData functions are handled too with latest updates.

Would you care to test the code I would post?

[wraithdu]

Sure I'd love to, especially using a buffer size that is smaller than the transfer size, ie requiring multiple chunk reads. That is where I was unable to get it to work without crashing.

[trancexx]

Great! This is example:

expandcollapse popup

#include "WinHttp.au3"


;XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Global $bChunked = True
;XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
; The size
Global $iSizeBufferAsync = 512 ; for example
; The buffer
Global $tBufferAsync



; Register Callback function
Global $hWINHTTP_STATUS_CALLBACK = DllCallbackRegister("__WINHTTP_STATUS_CALLBACK", "none", "handle;dword_ptr;dword;ptr;dword")

; Initialize and get session handle. Asynchronous flag.
Global $hOpen = _WinHttpOpen(Default, Default, Default, Default, $WINHTTP_FLAG_ASYNC)

; Assign callback function
_WinHttpSetStatusCallback($hOpen, $hWINHTTP_STATUS_CALLBACK)

; Get connection handle
Global $hConnect
If $bChunked Then
    $hConnect = _WinHttpConnect($hOpen, "google.com")
Else
    $hConnect = _WinHttpConnect($hOpen, "msdn.microsoft.com")
EndIf

; Make request
Global $hRequest = _WinHttpOpenRequest($hConnect)

; Send it
_WinHttpSendRequest($hRequest)

; Some dummy code for waiting
MsgBox(64 + 262144, "Wait...", "Wait for the results if they are not shown already.")


; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)
; Free callback. Redundant here
DllCallbackFree($hWINHTTP_STATUS_CALLBACK)



;XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
; Define callback function
;XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Func __WINHTTP_STATUS_CALLBACK($hInternet, $iContext, $iInternetStatus, $pStatusInformation, $iStatusInformationLength)
    #forceref $hInternet, $iContext, $pStatusInformation, $iStatusInformationLength
    ConsoleWrite(">> ")
    ; Interpret the status
    Local $sStatus
    Switch $iInternetStatus
        Case $WINHTTP_CALLBACK_STATUS_CLOSING_CONNECTION
            $sStatus = "Closing the connection to the server"
        Case $WINHTTP_CALLBACK_STATUS_CONNECTED_TO_SERVER
            $sStatus = "Successfully connected to the server."
        Case $WINHTTP_CALLBACK_STATUS_CONNECTING_TO_SERVER
            $sStatus = "Connecting to the server."
        Case $WINHTTP_CALLBACK_STATUS_CONNECTION_CLOSED
            $sStatus = "Successfully closed the connection to the server."
        Case $WINHTTP_CALLBACK_STATUS_DATA_AVAILABLE
            $sStatus = "Data is available to be retrieved with WinHttpReadData."
            ConsoleWrite($sStatus & @CRLF)

            ;*************************************
            ; Read asynchronously
            ;*************************************
            _WinHttpSimpleReadDataAsync($hInternet, DllStructGetPtr($tBufferAsync), DllStructGetSize($tBufferAsync))
            Return

        Case $WINHTTP_CALLBACK_STATUS_HANDLE_CREATED
            $sStatus = "An HINTERNET handle has been created: " & $hInternet
        Case $WINHTTP_CALLBACK_STATUS_HANDLE_CLOSING
            $sStatus = "This handle value has been terminated: " & $hInternet
        Case $WINHTTP_CALLBACK_STATUS_HEADERS_AVAILABLE
            $sStatus = "The response header has been received and is available with WinHttpQueryHeaders."
            ConsoleWrite($sStatus & @CRLF)

            ;*************************************
            ; Print header
            ;*************************************
            ConsoleWrite(_WinHttpQueryHeaders($hInternet) & @CRLF)

            Local $iSizeBuff = $iSizeBufferAsync ; _WinHttpQueryHeaders($hInternet, $WINHTTP_QUERY_CONTENT_LENGTH)
            If $iSizeBuff = 0 Then $iSizeBuff = 8192
            $tBufferAsync = DllStructCreate("byte[" & $iSizeBuff & "]")
            ;*************************************
            ; Check if there is any data available
            ;*************************************
            _WinHttpQueryDataAvailable($hInternet)

            Return

        Case $WINHTTP_CALLBACK_STATUS_INTERMEDIATE_RESPONSE
            $sStatus = "Received an intermediate (100 level) status code message from the server."
        Case $WINHTTP_CALLBACK_STATUS_NAME_RESOLVED
            $sStatus = "Successfully found the IP address of the server."
        Case $WINHTTP_CALLBACK_STATUS_READ_COMPLETE
            $sStatus = "Data was successfully read from the server."
            ConsoleWrite($sStatus & " In length of " & $iStatusInformationLength & " bytes" & @CRLF)

            ;*************************************
            ; Print read data
            ;*************************************
            If $iStatusInformationLength Then
                Local $sRead = DllStructGetData(DllStructCreate("char[" & $iStatusInformationLength & "]", $pStatusInformation), 1)
                ConsoleWrite($sRead & @CRLF)
                ; Read more
                _WinHttpSimpleReadDataAsync($hInternet, DllStructGetPtr($tBufferAsync), DllStructGetSize($tBufferAsync))
            EndIf
            Return

        Case $WINHTTP_CALLBACK_STATUS_RECEIVING_RESPONSE
            $sStatus = "Waiting for the server to respond to a request."
        Case $WINHTTP_CALLBACK_STATUS_REDIRECT
            $sStatus = "An HTTP request is about to automatically redirect the request."
        Case $WINHTTP_CALLBACK_STATUS_REQUEST_ERROR
            $sStatus = "An error occurred while sending an HTTP request."
        Case $WINHTTP_CALLBACK_STATUS_REQUEST_SENT
            $sStatus = "Successfully sent the information request to the server."
        Case $WINHTTP_CALLBACK_STATUS_RESOLVING_NAME
            $sStatus = "Looking up the IP address of a server name."
        Case $WINHTTP_CALLBACK_STATUS_RESPONSE_RECEIVED
            $sStatus = "Successfully received a response from the server."
        Case $WINHTTP_CALLBACK_STATUS_SECURE_FAILURE
            $sStatus = "One or more errors were encountered while retrieving a Secure Sockets Layer (SSL) certificate from the server."
        Case $WINHTTP_CALLBACK_STATUS_SENDING_REQUEST
            $sStatus = "Sending the information request to the server."
        Case $WINHTTP_CALLBACK_STATUS_SENDREQUEST_COMPLETE
            $sStatus = "The request completed successfully."
            ConsoleWrite($sStatus & @CRLF)

            ;*************************************
            ; Receive Response
            ;*************************************
            _WinHttpReceiveResponse($hInternet)
            Return

        Case $WINHTTP_CALLBACK_STATUS_WRITE_COMPLETE
            $sStatus = "Data was successfully written to the server."
    EndSwitch
    ; Print it
    ConsoleWrite($sStatus & @CRLF)
EndFunc

... and this is current dev version of WinHttp.au3.

Does that work for you? There is $bChunked variable in that example. If set to True it example reads google.com and if False msdn.microsoft.com. Former site uses chunked transfer encoding and latter specifies content length.

[wraithdu]

That's about what I came up with when I was testing, with the same basic results - 'AutoIt3 has stopped working...', or script crashing with variable not declared errors, etc. The usual stuff that happens when the callback system falls down.

Edit -

If it matters, I'm testing through an HTTP proxy at work (I've made the relevant HttpOpen modifications in your example).

Edited March 25, 2013 by wraithdu

[trancexx]

That's too bad. I got problems only if one callback is associated with more than one session.

Meh.

[trancexx]

Just quick answer to people who felt the need to tell me that my example has syntax error in line 74 and 115 when calling _WinHttpSimpleReadDataAsync()... there are no error there, the tool that you use to syntax-check has bug, not my code.

Edited March 25, 2013 by trancexx

[wraithdu]

This error is not valid? How can you have ByRef if you aren't calling with a variable?

error: _WinHttpSimpleReadDataAsync() called with Const or expression on ByRef-param(s).

[trancexx]

Some 4 years ago ByRef option was revised and that was added to AutoIt. It's all documented actually.