WinHTTP functions

[Kyan]

I think its "_WinHttpSendRequest" not "_WinHttpQueryDataAvailable" some time proxies consume too much time to send the request, specially the dead proxies; Try to set the timeout for request with this function "_WinHttpSetTimeouts".

no, its _WinHttpQueryHeaders(), since it sends all the data, and the function _WinHttpQueryDataAvailable return a true value (and before it, I check the ping..., but I'll look at "_WinHttpSetTimeouts" and see if I can set a timeout for query headers..

[Kyan]

what is the resolve timeout? (resolve server name/ip?)

[Chance]

Great UDF, I use it a lot.

And I've also found a problem with the function _WinHttpSendRequest, specifically the call to the WinHttpSendRequest DLL function.

I was using this to find out what was wrong. Long story short, it crashes when making the final call to WinHttpSendRequest, but only when using

particular proxies.

expandcollapse popup

Func _WinHttpSendRequest($hRequest, $sHeaders = Default, $sOptional = Default, $iTotalLength = Default, $iContext = Default)
    __WinHttpDefault($sHeaders, $WINHTTP_NO_ADDITIONAL_HEADERS)
    __WinHttpDefault($sOptional, $WINHTTP_NO_REQUEST_DATA)
    __WinHttpDefault($iTotalLength, 0)
    __WinHttpDefault($iContext, 0)
    Local $pOptional = 0, $iOptionalLength = 0
    If @NumParams > 2 Then
        Local $tOptional
        $iOptionalLength = BinaryLen($sOptional)
        $tOptional = DllStructCreate("byte[" & $iOptionalLength & "]")
        If $iOptionalLength Then $pOptional = DllStructGetPtr($tOptional)
        DllStructSetData($tOptional, 1, $sOptional)
    EndIf
    If Not $iTotalLength Or $iTotalLength < $iOptionalLength Then $iTotalLength += $iOptionalLength

    ConsoleWrite("handle:"& $hRequest & @CR & _
            "wstr:" & $sHeaders & @CR & _
            "dword:" & 0 & @CR & _
            "ptr:" & $pOptional & @CR & _
            "dword:" & $iOptionalLength & @CR & _
            "dword:" & $iTotalLength & @CR & _
            "dword_ptr:" & $iContext & @CR & "======" & @CR)

    Local $aCall = DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpSendRequest", _
            "handle", $hRequest, _
            "wstr", $sHeaders, _
            "dword", 0, _
            "ptr", $pOptional, _
            "dword", $iOptionalLength, _
            "dword", $iTotalLength, _
            "dword_ptr", $iContext)

    ConsoleWrite("+>Call was successful"&@CR)

    If @error Or Not $aCall[0] Then Return SetError(1, 0, 0)
    Return 1
EndFunc   ;==>_WinHttpSendRequest

Here is a reproducer of the anomaly.

Apparently, when using certain proxies, the call to the function fails and crashes the script, I don't know why, whic is why I'm here, but I do know that when using the proxy under system proxy, accessing a web resource with my browser results with no information returned and a page title reading "Action_Canceled".

expandcollapse popup

#include "WinHttp.au3"


$Test = _WebTest('66.0.2.9:443'); works flawlessly
ShowData( _
    "!> Error = " & @error & @CRLF & _
    "-> Extnd = " & @extended & @CRLF & _
    "+> Retrn = " & $Test & @CRLF & @CRLF _
)
MsgBox(0,"","Worked successfully")

$Test = _WebTest('134.37.254.27:80')
ShowData( _
    "!> Error = " & @error & @CRLF & _
    "-> Extnd = " & @extended & @CRLF & _
    "+> Retrn = " & $Test & @CRLF & @CRLF _
)
MsgBox(0,"","this message box will not appear because we will crash before reaching it at _WinHttpSendRequest($hRequest)")

Func _WebTest($Proxy = 0)
    Local $Error
    Local $hRequest
    Local $html

    Local $hRequest
    Local $Data

    Local $wHwnd = _WinHttpOpen("Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $Proxy)
    If @error Then Exit

    $hConnect = _WinHttpConnect($wHwnd, "www.google.com", $INTERNET_DEFAULT_HTTPS_PORT)
    If @error Then
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        _WinHttpCloseHandle($wHwnd)
        Return SetError(7, 0, "Failed to connect.")
    EndIf

    $hRequest = _WinHttpOpenRequest($hConnect, "GET", "nothing", "HTTP/1.1", "https://www.google.com/", Default, BitOR($WINHTTP_FLAG_SECURE, $WINHTTP_FLAG_ESCAPE_DISABLE))
    If Not $hRequest Then
        _WinHttpCloseHandle($wHwnd)
        _WinHttpCloseHandle($hRequest)
        Return SetError(8, 0, "Failed to open request.")
    EndIf

    ConsoleWrite("+>Starting"&@CR)

    _WinHttpSendRequest($hRequest);crashes here with certain proxies (I.E., 134.37.254.27:80, 80.247.33.46:80 and 134.37.254.27:8080)
    If @error Then
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        _WinHttpCloseHandle($wHwnd)
        Return SetError(9, 0, "Failed to send request.")
    EndIf

     ConsoleWrite("+>Escaped"&@CR)

    _WinHttpReceiveResponse($hRequest)
    If @error Then
        ;MsgBox(0,@ScriptLineNumber,"Error: "& @error & @CR & "ID = "& $ID[0])
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        _WinHttpCloseHandle($wHwnd)
        Return SetError(10, 0, "Failed to recieve response.")
    EndIf

    Local $Data = ''
    SetError(0)

    If _WinHttpQueryDataAvailable($hRequest) Then
        Do
            $Data &= _WinHttpReadData($hRequest, 0)
        Until @error
    Else
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        _WinHttpCloseHandle($wHwnd)
        Return SetError(11, 0, "No data.")
    EndIf

    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($wHwnd)

    Return SetError(0,0,$Data)
EndFunc

Func ShowData($Data)
    Local $UI = GUICreate("",700,400);to debug the response..
    GUICtrlCreateEdit($Data,0,0,699,399)
    GUISetState()
    While GUIGetMsg($UI) <> -3
        Sleep(100)
    WEnd
    GUIDelete($UI)
EndFunc

[trancexx]

I can't reproduce the issue.

Are you sure you have the right ports? You are using $INTERNET_DEFAULT_HTTPS_PORT no matter which one you specify together with IP.

What's your system?

Edited October 22, 2012 by trancexx

[Chance]

I can't reproduce the issue.

Are you sure you have the right ports? You are using $INTERNET_DEFAULT_HTTPS_PORT no matter which one you specify together with IP.

What's your system?

I'm using Windows XP x86 SP3, Intel Atom N280 CPU.

I also just tested this on a Windows Vista x64 SP2, didn't fail as it does on mine.

This is odd

Edit: Confirmed, this only happens on the type of laptop computer I was using to test it on, I tested it on another similar laptop, failed in the same was, but it doesn't fail when on the same laptop using windows 7, just windows xp... odd...

Edited October 24, 2012 by FlutterShy

[blityon]

Hello, I read and read ..... but my WinHttpOpenRequest https does not work or maybe I dont know it.

What happend?

It is a test https page.

A lot of thanks.

#include <WinHttp.au3>

$hOpen = _WinHttpOpen()
$hConnect = _WinHttpConnect($hOpen, "magroup-online.com", $INTERNET_DEFAULT_HTTPS_PORT)

$hRequest  = _WinHttpOpenRequest($hConnect, Default, "IBA/ES/ES/IBA_ES_es_PaymentSecurity.html", Default, Default, Default, $WINHTTP_FLAG_SECURE)

_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)

$sData = ""
If _WinHttpQueryDataAvailable($hRequest) Then
While 1
$sChunk = _WinHttpReadData($hRequest)
If @error Then ExitLoop
$sData &= $sChunk
WEnd
EndIf

$File = FileOpen("file.html", 2)
FileWriteLine($File, $sData)
FileClose($File)

_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

The result is: Nothing, an empty file

Edited November 2, 2012 by blityon

[wakillon]

Hello, I read and read ..... but my WinHttpOpenRequest https does not work or maybe I dont know it.

What happend?

It is a test https page.

A lot of thanks.

#include <WinHttp.au3>

$hOpen = _WinHttpOpen()
$hConnect = _WinHttpConnect($hOpen, "magroup-online.com", $INTERNET_DEFAULT_HTTPS_PORT)

$hRequest = _WinHttpOpenRequest($hConnect, Default, "IBA/ES/ES/IBA_ES_es_PaymentSecurity.html", Default, Default, Default, $WINHTTP_FLAG_SECURE)

_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)

$sData = ""
If _WinHttpQueryDataAvailable($hRequest) Then
While 1
$sChunk = _WinHttpReadData($hRequest)
If @error Then ExitLoop
$sData &= $sChunk
WEnd
EndIf

$File = FileOpen("file.html", 2)
FileWriteLine($File, $sData)
FileClose($File)

_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

The result is: Nothing, an empty file

try like this

#include <WinHttp.au3>

$sUrl = 'https://www.magroup-online.com/IBA/ES/ES/IBA_ES_es_PaymentSecurity.html'
$hOpen = _WinHttpOpen()
$aCrackedUrl = _WinHttpCrackUrl ( $sUrl )
$hConnect = _WinHttpConnect($hOpen, $aCrackedUrl[2], $INTERNET_DEFAULT_HTTPS_PORT)
$hRequest = _WinHttpOpenRequest($hConnect, Default, $aCrackedUrl[6], Default, Default, Default, $WINHTTP_FLAG_SECURE)
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)
$sData = ""
If _WinHttpQueryDataAvailable($hRequest) Then
    While 1
        $sChunk = _WinHttpReadData($hRequest)
        If @error Then ExitLoop
        $sData &= $sChunk
    WEnd
EndIf
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)
$File = FileOpen("file.html", 2)
FileWriteLine($File, $sData)
FileClose($File)

[blityon]

try like this

#include <WinHttp.au3>

$sUrl = 'https://www.magroup-online.com/IBA/ES/ES/IBA_ES_es_PaymentSecurity.html'
$hOpen = _WinHttpOpen()
$aCrackedUrl = _WinHttpCrackUrl ( $sUrl )
$hConnect = _WinHttpConnect($hOpen, $aCrackedUrl[2], $INTERNET_DEFAULT_HTTPS_PORT)
$hRequest = _WinHttpOpenRequest($hConnect, Default, $aCrackedUrl[6], Default, Default, Default, $WINHTTP_FLAG_SECURE)
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)
$sData = ""
If _WinHttpQueryDataAvailable($hRequest) Then
    While 1
        $sChunk = _WinHttpReadData($hRequest)
        If @error Then ExitLoop
        $sData &= $sChunk
    WEnd
EndIf
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)
$File = FileOpen("file.html", 2)
FileWriteLine($File, $sData)
FileClose($File)

Perfect, works great, many thanks!

[wakillon]

Perfect, works great, many thanks!

Don't forget to take a look to the _WinHttpCrackUrl Func !

[Chance]

I made this some time ago so I could just copy paste into things I make, helps with creating web stuff, thought I might share

Typically I copy this into something I'm making and all I have to deal with are the headers and additional data to send, like simulated forms stuffs.

I usually like maanaging them myself so that's why

Also, for anyone who don't know, you if on chome, hit "f12" > "network"> "the file you requested or post you made" and click on it, this lets you view all headers and post data, just copy and paste that to simulate your browser without cookies and javascript etc, some sites will know. Most don't.

Also, in the network tab in developers view, there's a button to record all activity at the bottom, make sure you click it if things redirect so stuff is saved.

expandcollapse popup

#include "WinHttp.au3"; you need WinHTTP
#Include "ZLIB.au3"; and ZLIB http://www.autoitscript.com/forum/topic/128962-zlib-deflateinflategzip-udf/

OnAutoItExitRegister("_TerminateSession")

Global $_WinHTTP_CurrentSession[2] = [ -1, -1]

$Return = _WinHTTP_Action("http://www.worldwideweb.com")
Switch @error
    Case 0
        ConsoleWrite( _
            "+=====HEADER====+" & @CRLF & _
            $Return[1] & @CRLF & _
            "-===== HTML ====-" & @CRLF & _
            BinaryToString($Return[0]) & @CRLF _
        )
    Case 1
        MsgBox(48, "Error!", "WinHTTP is not available on your system!")

    Case 2
        MsgBox(16, "Error!", "You need to supply a URL :P")

    Case 3
        MsgBox(16, "Error!", "Error cracking URL! URL might be invalid.")

    Case 4
        MsgBox(16, "Error!", "Error creating session!")

    Case 5
        MsgBox(16, "Error!", "Error in connecting to website!")

    Case 6
        MsgBox(16, "Error!", "Error in creating request!")

    Case 7
        MsgBox(16, "Error!", "Unable to send request correctly.")

    Case 8
        MsgBox(16, "Error!", "Website did not respond.")

    Case 9
        MsgBox(16, "Error!", "No data was available from the request :/")

EndSwitch
Exit


; #FUNCTION# ====================================================================================================================
; Name ..........: _WinHTTP_Action
; Description ...:
; Syntax ........: _WinHTTP_Action($Page[, $Action = Default[, $bBinary = Default[, $sUserAgent = Default[, $headers = Default[,
;                  $AdditionalData = Default[, $sReferer = Default[, $ReadReturn = Default[, $Proxy = Default[,
;                  $SessionDur = Default]]]]]]]]])
; Parameters ....: $Page                - String value of internet resource
;
;                  $Action              - [optional] HTTP verb to use in the request. Default is "GET".
;
;                  $bBinary             - [optional] A bool value. Default is False, which returns data as text.
;                                            | True returns binary, such as a file on the internet, like an exe.
;
;                  $sUserAgent          - [optional] A string value. Default is a GoogleBot user-agent string.
;                                            | A GoogleBot user-agent is used because sometimes it allows you to bypass
;                                            | a websites login requests, sites do this to optimize websearch results for their
;                                            | sites so we take advantage of this little trick.
;
;                  $headers             - [optional] A string value. Default is to accept all, keep alive and request
;                                           | compressed gzip encoding to reduce load.
;
;                  $AdditionalData      - [optional] A string value. Default is nothing, this is where you typically send
;                                             | the target server some arguments/variables or other stuff.
;
;                  $sReferer            - [optional] A string value. Default is nothing I think.
;
;                  $ReadReturn          - [optional] An bool value. Default is True and will return response headers and
;                                            | data in an array. False will just make the POST or GET request and come back.
;
;                  $Proxy               - [optional] A string value. Default is not to use a proxy, if you want to use a proxy
;                                            | then just put it in like "123.123.123.123:54321".
;
;                  $SessionDur          - [optional] An int value. Default is 60000 miliseconds which is 60 seconds.
;                                            | this is used to tell the function how long this session is supposed to last
;                                            | while this function is not called for an extended time.
;                                            | is this value is zero (0), the session is terminated upon compleation...
;
; Return values .: An array.
;                    0|HTML
;                    1|Responce Headers
;
; Author ........: ScriptKitty
; Remarks .......: This function will automatically handle compressed data, charset type, session, http/https access
; Example .......: No
; ===============================================================================================================================
Func _WinHTTP_Action( _
        $Page, _
        $Action = Default, _
        $bBinary = Default, _
        $sUserAgent = Default, _
        $headers = Default, _
        $AdditionalData = Default, _
        $sReferer = Default, _
        $ReadReturn = Default, _
        $Proxy = Default, _
        $SessionDur = Default _
        )

    Local $DefHeaders = _
            "Connection: Keep-alive" & @CRLF & _
            "Accept: */*" & @CRLF & _
            "Accept-Encoding: gzip;q=0.9,*;q=0" & @CRLF

    __WinHttpDefault($Action, "GET")
    __WinHttpDefault($bBinary, False)
    __WinHttpDefault($sUserAgent, "Googlebot/2.1 (+http://www.google.com/bot.html)")
    __WinHttpDefault($headers, $DefHeaders)
    __WinHttpDefault($ReadReturn, True)
    __WinHttpDefault($SessionDur, 60000)
    __WinHttpDefault($Proxy, False)

$Action = StringUpper($Action)

    If ($_WinHTTP_CurrentSession[1] <> 1) Then
        If Not _WinHttpCheckPlatform() Then Return SetError(1, 0, 0)
        $_WinHTTP_CurrentSession[1] = 1
    EndIf

    If Not $Page Then Return SetError(2, 0, 0)

    Local $Crack = _WinHttpCrackUrl($Page)
    If @error Then Return SetError(3, 0, 0)

    Local $Port = $INTERNET_DEFAULT_PORT
    Local $Flag
    Local $iCharset = 0

    If Not $Crack[0] Then $Crack[0] = "https"

    If StringLower($Crack[0]) == "http" Then
        $Port = $INTERNET_DEFAULT_HTTP_PORT
        $Flag = Default
    ElseIf StringLower($Crack[0]) == "https" Then
        $Port = $INTERNET_DEFAULT_HTTPS_PORT
        $Flag = $WINHTTP_FLAG_SECURE
    EndIf

    If ($_WinHTTP_CurrentSession[0] = -1) Then
        If $Proxy Then
            $_WinHTTP_CurrentSession[0] = _WinHttpOpen($sUserAgent, $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $Proxy)
        Else
            $_WinHTTP_CurrentSession[0] = _WinHttpOpen($sUserAgent)
        EndIf
        If @error Then
            $_WinHTTP_CurrentSession[0] = -1
            Return SetError(4, 0, 0)
        EndIf
    EndIf

    AdlibRegister("_TerminateSession", $SessionDur)

    Local $hConnect = _WinHttpConnect($_WinHTTP_CurrentSession[0], $Crack[2], $Port)
    If @error Then
        _WinHttpCloseHandle($hConnect)
        Return SetError(5, 0, 0)
    EndIf

    Local $hRequest = _WinHttpOpenRequest($hConnect, $Action, $Crack[6] & $Crack[7], Default, $sReferer, Default, $Flag)
    If Not $hRequest Then
        _WinHttpCloseHandle($hConnect)
        Return SetError(6, 0, 0)
    EndIf

    If $Action == "POST" Then
        If ($headers = Default) And ($AdditionalData = Default) Then _WinHttpAddRequestHeaders($hRequest, "Content-Type: application/x-www-form-urlencoded")
    EndIf

    Local $iSize = 0
    If $AdditionalData Then $iSize = StringLen($AdditionalData)

    _WinHttpSetOption($hRequest, $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REDIRECT_POLICY_ALWAYS); allow redirects

    _WinHttpSendRequest($hRequest, $headers, $AdditionalData, $iSize)
    If @error Then
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        Return SetError(7, 0, 0)
    EndIf

    _WinHttpReceiveResponse($hRequest)
    If @error Then
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        Return SetError(8, 0, 0)
    EndIf

    If Not $ReadReturn Then
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        Return SetError(0, 0, 1)
    EndIf

    Local $Data
    If _WinHttpQueryDataAvailable($hRequest) Then
        Local $bChunk
        While 1
            $bChunk = _WinHttpReadData($hRequest, 2)
            If @error Then ExitLoop
            $Data = _WinHttpBinaryConcat($Data, $bChunk)
        WEnd
    Else
        _WinHttpCloseHandle($hRequest)
        _WinHttpCloseHandle($hConnect)
        Return SetError(9, 0, 0)
    EndIf

    Local $aReturn[2] = [$Data, _WinHttpQueryHeaders($hRequest)]

    If StringRegExp($aReturn[1], "(?im)^Content-Type:h.*?charseth*=h*utf-?8") Then $iCharset = 4
    If StringRegExp($aReturn[1], "(?im)^Content-Encoding:h+gzip") Then
        $Data = _ZLIB_GZUncompress($aReturn[0])
        If $bBinary Then
            $aReturn[0] = Binary($Data)
        Else
            $aReturn[0] = BinaryToString($Data, $iCharset)
        EndIf
    Else
        If $bBinary Then
            $aReturn[0] = Binary($Data)
        Else
            $aReturn[0] = BinaryToString($aReturn[0], $iCharset)
        EndIf
    EndIf

    If $SessionDur = 0 Then _TerminateSession()

    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    Return SetError(0, 0, $aReturn)
EndFunc   ;==>_WinHTTP_Action

; #FUNCTION# ====================================================================================================================
; Name ..........: _TerminateSession
; Description ...: Terminates a session after it's idled for a while.
; Syntax ........: _TerminateSession()
; Parameters ....: None
; Return values .: None
; Author ........: ScriptKitty
; Example .......: No
; ===============================================================================================================================
Func _TerminateSession()
    AdlibUnRegister("_TerminateSession")
    If ($_WinHTTP_CurrentSession[0] <> -1) Then
        _WinHttpCloseHandle($_WinHTTP_CurrentSession[0])
        $_WinHTTP_CurrentSession[0] = -1
    EndIf
EndFunc

Edited December 3, 2012 by FlutterShy

[water]

Two typos I found:

Case 3: MsgBox(16, "Error!", "Error cracking URL! URL might be invlaid.")

and

Case 9: MsgBox(16, "Error!", "No data was avaiable from the request :/")

[Chance]

I'm a very bad speller to begin with lol

thanks

[water]

As a non native speaker I'm a bad speller too

[Chance]

Just thought I might add this here, I know there are a lot of functions already for this but I don't think there's any here using this API.

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name...........: _WinAPI_UrlEscape
; Description....: Encodes data for safe use in URL quary strings.
; Syntax.........: _WinAPI_UrlEscape( $sString[, $iFlags])
; Parameters.....: $sString   - The string to convert.
;                  $iFlags - The flags that specify how the string is handled.
;                  Follow MSDN link at bottom to learn what combinations work.
;                  Default is to convert everything that is unsafe, like spaces and special characters.
;
;                  $URL_ESCAPE_SPACES_ONLY          = (0x04000000)
;                  $URL_ESCAPE_PERCENT              = (0x00001000)
;                  $URL_ESCAPE_AS_UTF8              = (0x00040000)
;                  $URL_DONT_ESCAPE_EXTRA_INFO      = (0x02000000)
;                  $URL_ESCAPE_SEGMENT_ONLY         = (0x00002000)
;                  $URL_ESCAPE_ASCII_URI_COMPONENT  = (0x00080000)
;
; Return values..: Success - The converted URL.
;                  Failure - Empty string and sets the @error flag to non-zero
;                  1| the dll call failed
;                  2| some other error, @extended flag contains the system error code.
;
; Remarks........: By default, UrlEscape ignores any text following a # or ? character. The URL_ESCAPE_SEGMENT_ONLY flag
;                  overrides this behavior by regarding the entire string as the segment. The URL_ESCAPE_SPACES_ONLY flag
;                  overrides this behavior, but only for space characters.
; Related........:
; Link...........: http://msdn.microsoft.com/en-us/library/windows/desktop/bb773774(v=vs.85).aspx
;
; Example........:  _WinAPI_UrlEscape("^%&#[]{}/<>|`""+=-_.,'~" & @TAB & " ")
;                   Results in: %5E%%26%23%5B%5D%7B%7D%2F%5C%3C%3E%7C%60%22+=-_.,'~%09%20
; ===============================================================================================================================

Func _WinAPI_UrlEscape($sString, $iFlags = 0x00002000)

    Local $aResult = DllCall("Shlwapi.dll", "long", "UrlEscapeW", "wstr", $sString,  "wstr", 0, "dword*", 2083, "dword", $iFlags)

    If @error Then Return SetError(1, 0, '')
    If $aResult[0] Then Return SetError(2, $aResult[0], '')
    Return SetError(0, 0, $aResult[2])
EndFunc   ;==>_WinAPI_UrlEscape

[wakillon]

Just thought I might add this here, I know there are a lot of functions already for this but I don't think there's any here using this API.

Take a look to of Yashield

[Chance]

Take a look to of Yashield

I noticed it has a function called CanonicalizeURL or something, but I don't think it has a UDF using that API

And I don't think it's been posted anywhere on this website either..

---

On another subject~

Some time ago I was messing around in C++ trying to make a little program that uses WinInet to make POST requests to a server, before successfully creating that little program, I made a post in the dev forum asking a question about why the wininet call was not being compleated unless I added a sleep in there, I later found out that it was because I was using a flag "FLAG_ASYNC" or something like that which caused the operaion to happen in the background, then I found out that win http has this flag option too.

I'm trying to use it, but apparantly there are some problems.

What I'm trying to do is create a script that will open around 7 through 10 requests to multiple servers at the same time, I found out that you cant do it with the callback as explained in this thread further back, so I tried doing it without a callback and just managing the sessions through an array.

The problem is that success seems to be pretty random, I'm using different proxies through all the requests and all of them work as of this post.

But what's happening is that _WinHttpReadData() seems to be working only when it feels like it, I created this script to demonstrate what's wrong in another larger project.

This example will deliver the headers most of the time, but may or may not recieve the HTML the server responded with, the server I'm making this request to will respond with something like "IP: xxx.xxx.xxx.xxx".

expandcollapse popup

#AutoIt3Wrapper_AU3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

Opt("MustDeclareVars", 1)

#include "WinHttp.au3"

;Info      : What this script will do is open to requests to a website that will shoow you your IP address, from my location.
;            - the majority of these proxies all work as of the date of this post.
;Problem   : Most of the time, the request headers are read correctly, but the responce HTML is rarely read ever...
;more info : If you use the proxy on another script without WINHTTP_FLAG_ASYNC paramater, all these proxies typically work and revieve the HTML and headers
;            - but for some reason in WINHTTP_FLAG_ASYNC mode, the HTML will rarely be recieved, while headers always come through with no problems.

Global $Proxies[10] = [ _
        "186.192.7.134:8080", _
        "202.150.130.82:8080", _
        "202.131.109.66:3128", _
        "177.43.118.10:3128", _
        "37.32.29.33:3128", _
        "189.41.84.104:3128", _
        "190.114.224.20:80", _
        "200.21.193.98:3128", _
        "46.209.66.1:3128", _
        "103.4.146.139:8080" _
        ]

Global $RecievedData[10][3]

Global $Index, $Data, $Status

Global $hOpen[10][3]
Global $hConnect[10]
Global $hRequest[10]

If MsgBox(4, "Information!", "Press ""Yes"" to simultaniously open 10 HTTP sessions in Async mode....") <> 6 Then Exit

ConsoleWrite(@CR)
ConsoleWrite("->===============================================================================================================================" & @CR)

For $I = 0 To 9

    $hOpen[$I][0] = _WinHttpOpen("(Compatable; AutoItv3 Async Test)", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $Proxies[$I], Default, $WINHTTP_FLAG_ASYNC)
    $hOpen[$I][1] = True
    $hOpen[$I][2] = $Proxies[$I]

    _WinHttpSetTimeouts($hOpen[$I][0], 0, 3000, 3000, 3000)
    _WinHttpSetOption($hOpen[$I][0], $WINHTTP_OPTION_REDIRECT_POLICY, $WINHTTP_OPTION_REDIRECT_POLICY_ALWAYS); allow redirects

    ConsoleWrite("+>Opened Session " & ($I + 1) & " with " & $Proxies[$I])

    $hConnect[$I] = _WinHttpConnect($hOpen[$I][0], 'unrealx.lt', $INTERNET_DEFAULT_HTTP_PORT)
    If @error Then
        _WinHttpCloseHandle($hOpen[$I][0])
        ConsoleWrite(" Error: Closed Open Handle!" & $Proxies[$I] & @CR)
        ContinueLoop
    EndIf

    $hRequest[$I] = _WinHttpOpenRequest($hConnect[$I], Default, 'showip.php', Default, Default, Default)
    If Not $hRequest[$I] Then
        _WinHttpCloseHandle($hConnect[$I])
        _WinHttpCloseHandle($hOpen[$I][0])
        ConsoleWrite(" Error: Closed All Handles!" & $Proxies[$I] & @CR)
        ContinueLoop
    EndIf

    _WinHttpSendRequest($hRequest[$I])
    ConsoleWrite(@CR)
Next

ConsoleWrite("->===============================================================================================================================" & @CR )

For $I = 0 To 999999
    $Index = Mod($I, 10)
    If $hOpen[$Index][1] Then

        _WinHttpReceiveResponse($hRequest[$Index])
        $Status = _WinHttpQueryDataAvailableEx($hRequest[$Index])
        If @error = $ERROR_WINHTTP_INCORRECT_HANDLE_STATE Then
            _WinHttpCloseHandle($hRequest[$Index])
            _WinHttpCloseHandle($hConnect[$Index])
            _WinHttpCloseHandle($hOpen[$Index][0])

            $RecievedData[$Index][0] = "Error!"
            $RecievedData[$Index][1] = "Error!"
            $RecievedData[$Index][2] = $hOpen[$Index][2]

            $hOpen[$Index][1] = False
            $Data = 0
        ElseIf @error = -9001 Then
            MsgBox(0, "", "-9001")

        ElseIf $Status Then

            Do
                $Data &= _WinHttpReadData($hRequest[$Index], 0)
            Until @error

            $RecievedData[$Index][0] = StringRegExpReplace($Data, "r*n*", "")
            $RecievedData[$Index][1] = StringRegExpReplace(_WinHttpQueryHeaders($hRequest[$Index]), "(?:rn|)([^r]+)rn", "[$1] ")
            $RecievedData[$Index][2] = $hOpen[$Index][2]

            _WinHttpCloseHandle($hRequest[$Index])
            _WinHttpCloseHandle($hConnect[$Index])
            _WinHttpCloseHandle($hOpen[$Index][0])

            $hOpen[$Index][1] = False
            $Data = ''

        Else
            If MsgBox(4, "Current Session: " & $Index, "Session not yet complete..." & @CR & "Check next?") <> 6 Then ExitLoop
        EndIf
    Else
        If MsgBox(4, "Cur Index: " & $Index, "This session was completed and processed..." & @CR & "Continue to next?") <> 6 Then ExitLoop
    EndIf
Next

If MsgBox(4, "Question...", "Display Results?") = 6 Then
ConsoleWrite(@CR)
    For $I = 0 To 9
        ConsoleWrite(">" & ($I + 1) & "#=============================================================================================================================" & @CR)
        ConsoleWrite("+>Proxy     : " & $RecievedData[$I][2] & @CRLF)
        ConsoleWrite("+>Headers   : " & $RecievedData[$I][1] & @CRLF)
        ConsoleWrite("+>HTML      : " & $RecievedData[$I][0] & @CRLF)
        ConsoleWrite(">===============================================================================================================================" & @CR & @CR)
    Next
    ConsoleWrite(@CR)
EndIf

For $I = 0 To 9
    _WinHttpCloseHandle($hRequest[$I])
    _WinHttpCloseHandle($hConnect[$I])
    _WinHttpCloseHandle($hOpen[$I][0])
Next

Func _WinHttpQueryDataAvailableEx($hRequest); I need error code here
    Local $aCall = DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpQueryDataAvailable", "handle", $hRequest, "dword*", 0)
    If @error Then Return SetError(-9001, 0, 0)
    Local $aResult = DllCall("kernel32.dll", "dword", "GetLastError")
    Return SetError($aResult[0], $aCall[2], $aCall[0])
EndFunc   ;==>_WinHttpQueryDataAvailableEx

[wakillon]

I noticed it has a function called CanonicalizeURL or something, but I don't think it has a UDF using that API

And I don't think it's been posted anywhere on this website either..

It was in a previous WinAPIEx.au3 of Yashied

- Version: 3.6 / 3.3.8.0

+ Version: 3.7 / 3.3.6.1

Don't know why it was removed in the last...

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name...........: _WinAPI_UrlEscape
; Description....: Converts characters in a URL that might be altered during transport across the Internet into their corresponding escape sequences.
; Syntax.........: _WinAPI_UrlEscape ( $sUrl [, $iFlags] )
; Parameters.....: $sUrl - The URL.
; $iFlags - The flags that indicate which characters in the URL should be converted to their escape sequences
; (an escape sequence has the form "%xy"). It can be a combination of the following values.
;
; $URL_DONT_ESCAPE_EXTRA_INFO
; $URL_ESCAPE_PERCENT
; $URL_ESCAPE_SEGMENT_ONLY
; $URL_ESCAPE_SPACES_ONLY

; Windows 7 or later
;
; $URL_ESCAPE_AS_UTF8
;
; Return values..: Success - The escaped URL.
; Failure - Empty string and sets the @error flag to non-zero, @extended flag may contain the system error code.
; Author.........: Yashied
; Modified.......:
; Remarks........: None
; Related........:
; Link...........: @@MsdnLink@@ UrlEscape
; Example........: Yes
; ===============================================================================================================================

Func _WinAPI_UrlEscape($sUrl, $iFlags = 0)
    Local $Ret = DllCall('shlwapi.dll', 'uint', 'UrlEscapeW', 'wstr', $sUrl, 'wstr', '', 'dword*', 4096, 'dword', $iFlags)
    If @error Then
        Return SetError(1, 0, '')
    Else
        If $Ret[0] Then
            Return SetError(1, $Ret[0], '')
        EndIf
    EndIf
    Return $Ret[2]
EndFunc ;==>_WinAPI_UrlEscape

A simple google search with autoit+UrlEscapeW would have shown you...

[Chance]

It was in a previous WinAPIEx.au3 of Yashied

- Version: 3.6 / 3.3.8.0

+ Version: 3.7 / 3.3.6.1

Don't know why it was removed in the last...

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name...........: _WinAPI_UrlEscape
; Description....: Converts characters in a URL that might be altered during transport across the Internet into their corresponding escape sequences.
; Syntax.........: _WinAPI_UrlEscape ( $sUrl [, $iFlags] )
; Parameters.....: $sUrl - The URL.
; $iFlags - The flags that indicate which characters in the URL should be converted to their escape sequences
; (an escape sequence has the form "%xy"). It can be a combination of the following values.
;
; $URL_DONT_ESCAPE_EXTRA_INFO
; $URL_ESCAPE_PERCENT
; $URL_ESCAPE_SEGMENT_ONLY
; $URL_ESCAPE_SPACES_ONLY

; Windows 7 or later
;
; $URL_ESCAPE_AS_UTF8
;
; Return values..: Success - The escaped URL.
; Failure - Empty string and sets the @error flag to non-zero, @extended flag may contain the system error code.
; Author.........: Yashied
; Modified.......:
; Remarks........: None
; Related........:
; Link...........: @@MsdnLink@@ UrlEscape
; Example........: Yes
; ===============================================================================================================================

Func _WinAPI_UrlEscape($sUrl, $iFlags = 0)
Local $Ret = DllCall('shlwapi.dll', 'uint', 'UrlEscapeW', 'wstr', $sUrl, 'wstr', '', 'dword*', 4096, 'dword', $iFlags)
If @error Then
Return SetError(1, 0, '')
Else
If $Ret[0] Then
Return SetError(1, $Ret[0], '')
EndIf
EndIf
Return $Ret[2]
EndFunc ;==>_WinAPI_UrlEscape

A simple google search with autoit+UrlEscapeW would have shown you...

Well, I stand corrected

And I did search it on google like "site:autoitscript.com UrlEscape", but I duno why I didn't find it the first time...

[Draygoes]

Not bad! I am sortof old school when it comes to autoit (Havent used it in a while) so this set of UDF's will make life much nicer.

[trancexx]

I have made some changes to the code, nothing big. Fixed a bug with form filling function regarding application/x-www-form-urlencoded enc. type, switched to some new DllCall() types and gave the help file new fresh look.

Anyone interested can find download link in the first post of this thread. Minimum AutoIt version is 3.3.7.20.