Jump to content

winhttp get file name problem

youtuber

By youtuber,
in AutoIt General Help and Support

 Share

Recommended Posts

youtuber Universalist

youtuber

Posted
Posted (edited)

I can't get file names in any way 😟 

Global $array1[4] = ["https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe" & @CRLF & _
        "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream" & @CRLF & _
        "https://www.grigsoft.com/wincmp-setup.zip" & @CRLF & _
        "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1" & @CRLF & _
        "https://www.7-zip.org/a/7z1900-x64.exe"]

For $i = 0 To UBound($array1) - 1
    $ReadUrl = StringRegExp($array1[0], '(?i)(?:https?://|www\.)+[\w.?+=&%@#!:\-/]+\w', 3)
    If IsArray($ReadUrl) Then
        ConsoleWrite(_HTTPGetFileName($ReadUrl[$i]) & @CRLF)
    EndIf
Next

Func _HTTPGetFileName($sUrl)
    $oHTTP = ObjCreate('winhttp.winhttprequest.5.1')
    $oHTTP.Open("HEAD", $sUrl, False)
    $oHTTP.SetRequestHeader('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36')
    $oHTTP.Send()
    $oHTTP.WaitForResponse
    $ContentDisposition = $oHTTP.GetAllResponseHeaders()
    $array = StringRegExp($ContentDisposition, 'filename="(.*)"', 3)
    ;$array = StringRegExpReplace($ContentDisposition, '(?si).*filename="(\S+)".*', '$1')
    If IsArray($array) Then Return $array[0]
EndFunc   ;==>HTTPFileName

 

Edited by youtuber
Link to comment
Share on other sites
youtuber Universalist

youtuber

Posted
  • Author
Posted

@Danp2 It doesn't work for me, unfortunately I've tried them before. 

#include "WinHttp.au3"

Global $sURL = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe"
Global $aCracked = _WinHttpCrackUrl($sURL)

Global $hHttpOpen = _WinHttpOpen()
Global $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
Global $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
_WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
_WinHttpSendRequest($hRequest)
_WinHttpReceiveResponse($hRequest)

Global $sDisp = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_DISPOSITION)
ConsoleWrite($sDisp & @CRLF)

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hHttpOpen)

http://prntscr.com/oskavr

Link to comment
Share on other sites
youtuber Universalist

youtuber

Posted
  • Author
Posted (edited)

@Deye This is not possible, it will always fail because file names are not fixed :) 

Global $array1[4] = ["https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe" & @CRLF & _
        "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream" & @CRLF & _
        "https://www.grigsoft.com/wincmp-setup.zip" & @CRLF & _
        "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1" & @CRLF & _
        "https://www.7-zip.org/a/7z1900-x64.exe"]

$types = "(?=.exe|.zip)"

For $i = 0 To UBound($array1) - 1
    $ReadUrl = StringRegExp($array1[0],  "[\w -]*+" & $types & ".\w{3}", 3)
    If IsArray($ReadUrl) Then
        ConsoleWrite(URLDecode($ReadUrl[$i]) & @CRLF)
    EndIf
Next

Func URLDecode($urlText)
    $urlText = StringReplace($urlText, "+", " ")
    Local $matches = StringRegExp($urlText, "\%([abcdefABCDEF0-9]{2})", 3)
    If Not @error Then
        For $match In $matches
            $urlText = StringReplace($urlText, "%" & $match, BinaryToString('0x' & $match))
        Next
    EndIf
    Return $urlText
EndFunc   ;==>URLDecode

console output
20Setup.exe
95-win32.zip
wincmp-setup.zip
autoit-winhttp/zip

or 

Dim $StringSplit
$URL = "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream"

$StringSplit = StringSplit($URL, "/")
$FileName = $StringSplit[UBound($StringSplit)-1]

ConsoleWrite("File Name : " & URLDecode($FileName) & @CRLF)

Func URLDecode($urlText)
    $urlText = StringReplace($urlText, "+", " ")
    Local $matches = StringRegExp($urlText, "\%([abcdefABCDEF0-9]{2})", 3)
    If Not @error Then
        For $match In $matches
            $urlText = StringReplace($urlText, "%" & $match, BinaryToString('0x' & $match))
        Next
    EndIf
    Return $urlText
EndFunc   ;==>URLDecode

console output

File Name : eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20190730/us-east-1/s3/aws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment; filename=upx-3.95-win32.zip&response-content-type=application/octet-stream?
 

:'(

Edited by youtuber
Link to comment
Share on other sites
Deye Universalist

Deye

Posted
Posted

I'm not sure i understand, what do you get out trying this ?

Spoiler 
#include <File.au3>
Global $string = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe" & @CRLF & _
        "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream" & @CRLF & _
        "https://www.grigsoft.com/wincmp-setup.zip" & @CRLF & _
        "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1" & @CRLF & _
        "https://www.7-zip.org/a/7z1900-x64.exe"


$types = "(?=.exe|.zip)"
$ReadUrl = StringRegExp($string, "[%\w -]*+" & $types & "\.\w{3}", 3)
For $i = 0 To UBound($ReadUrl) - 1
    If StringInStr($ReadUrl[$i], "%") Then
        $ReadUrl[$i] = StringTrimLeft($ReadUrl[$i], 4)
    EndIf
Next
_ArrayDisplay($ReadUrl)

 

 

Link to comment
Share on other sites
KaFu Universalist

KaFu

Posted
Posted (edited)

There are several possible ways a download can get it's filename, I guess you'll have to test it one by one, response header might have prio though, and I guess content-disposition and location must not be used in the same header

https://stackoverflow.com/questions/3102226/how-to-set-name-of-file-downloaded-from-browser

By Response Header
    Content-Disposition: attachment; filename=autoit-winhttp-1.6.4.1.zip
    >> from filename
    OR
    Location: http://www.funk.eu/downloads/hmw/HMW_v3.exe.zip
    >> from Location URL

By URL
    Last Part of URL
    
By HTML5 "download" Attribute

 

Edit:

In your example above e.g. response-content-disposition=attachment; filename=upx-3.95-win32.zip

Edited by KaFu

de.png OS: Win10-22H2 - 64bit - German, AutoIt Version: 3.3.16.1, AutoIt Editor: SciTE, Website: https://funk.eu
prog_icon_amt.png AMT - Auto-Movie-Thumbnailer (2024-Oct-13) prog_icon_bic.png BIC - Batch-Image-Cropper (2023-Apr-01) prog_icon_cop.png COP - Color Picker (2009-May-21)
prog_icon_dcs.png DCS - Dynamic Cursor Selector (2024-Oct-13) prog_icon_hmw.png HMW - Hide my Windows (2024-Oct-19) prog_icon_hrc.png HRC - HotKey Resolution Changer (2012-May-16) 
prog_icon_icu.png ICU - Icon Configuration Utility (2018-Sep-16) prog_icon_smf.png SMF - Search my Files (2024-Oct-20) - THE file info and duplicates search tool prog_icon_ssd.png SSD - Set Sound Device (2017-Sep-16)

Link to comment
Share on other sites
mikell Universalist

mikell

Posted
Posted
3 hours ago, youtuber said:

@mikell if there is no name in the url if I encounter a different url?

So as Kafu said and as you mentioned in post #1, get the headers
Personally as soon as it begins to be tricky with redirections and so on, I use Curl which makes the whole job for me  :)

#include <AutoItConstants.au3>

$link = "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1"
Local $iPID = Run("curl -Lsk --head " & $link, "", @SW_HIDE, $STDOUT_CHILD)
ProcessWaitClose($iPID)
$filename = StringRegExpReplace(StdoutRead($iPID), '(?s).*filename=(\S+).*', '$1')
ConsoleWrite($filename & @crlf)

To visualize the headers, ConsoleWrite on StdoutRead

Link to comment
Share on other sites
youtuber Universalist

youtuber

Posted
  • Author
Posted

@mikell I tried your example I downloaded Curl this didn't work for me.
Did not return any value.  :(

https://prnt.sc/osq1ss 

#include <AutoItConstants.au3>

$link = "http://mik.dyndns.pro/dos-stuff/bin/curl764.7z"
Local $iPID = Run("curl -Lsk --head " & $link, "", @SW_HIDE, $STDOUT_CHILD)
ProcessWaitClose($iPID)
$filename = StringRegExpReplace(StdoutRead($iPID), '(?s).*filename=(\S+).*', '$1')
ConsoleWrite($filename & @crlf)

 

Link to comment
Share on other sites
Deye Universalist

Deye

Posted
Posted

ok just had another look, think i understood now

maybe  something like this should work ?

you will need to add all cases for  numbers perfixed with "%" like Chr(37) & "3" and all file types you need to grab in 

#include <File.au3>

Global $array1[4] = ["https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe" & @CRLF & _
        "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream" & @CRLF & _
        "https://www.grigsoft.com/wincmp-setup.zip" & @CRLF & _
        "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1" & @CRLF & _
        "https://www.7-zip.org/a/7z1900-x64.exe"]
$types = "(?=.exe|.zip)"

For $i = 0 To UBound($array1) - 1
    $ReadUrl = StringRegExp($array1[$i], "[\w -.%]+" & $types & "\.\w{3}", 3)
    For $j = 0 To UBound($ReadUrl) - 1
        If StringInStr($ReadUrl[$j], Chr(37) & "3") Then
            $ReadUrl[$j] = StringTrimLeft($ReadUrl[$j], StringInStr($ReadUrl[$j], Chr(37) & "3", 0, -1) + 1)
        EndIf
        $ReadUrl[$j] = StringReplace($ReadUrl[$j], Chr(37) & "20", "")
    Next
    _ArrayDisplay($ReadUrl)
Next

 

Link to comment
Share on other sites
KaFu Universalist

KaFu

Posted
Posted (edited) 
#include "WinHttp.au3"
#include <array.au3>

Global $array1[5]

$array1[0] = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe"
$array1[1] = "https://web.archive.org/web/20190730142339if_/https://github-production-release-asset-2e65be.s3.amazonaws.com/67031040/eb6ad800-a8f2-11e8-9ab0-042afa1a979a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190730%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190730T142323Z&X-Amz-Expires=300&X-Amz-Signature=c64ccf4eae74a747e79c4f26524722842d6c1e9eba71605b20c4fc0406c11933&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dupx-3.95-win32.zip&response-content-type=application%2Foctet-stream"
$array1[2] = "https://www.grigsoft.com/wincmp-setup.zip"
$array1[3] = "https://codeload.github.com/dragana-r/autoit-winhttp/zip/1.6.4.1"
$array1[4] = "https://www.7-zip.org/a/7z1900-x64.exe"

For $i = 0 To UBound($array1) - 1

    $aCracked = _WinHttpCrackUrl($array1[$i])
    ; _ArrayDisplay($aCracked)

    $hHttpOpen = _WinHttpOpen()
    $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
    $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
    _WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
    _WinHttpSendRequest($hRequest)
    _WinHttpReceiveResponse($hRequest)

    $sDisp = _WinHttpQueryHeaders($hRequest)
    ; ConsoleWrite($sDisp & @CRLF)

    Local $sFilename
    If StringInStr($sDisp, 'Content-Disposition: attachment; filename=') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Content-Disposition: attachment; filename=') Then
                $sFilename = StringReplace($aDisp[$y], 'Content-Disposition: attachment; filename=', '')
                ExitLoop
            EndIf
        Next
    ElseIf StringInStr($sDisp, 'Location: ') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Location: ') Then
                $sFilename = StringReplace($aDisp[$y], 'Location: ', '')
                $sFilename = StringRight($sFilename, StringLen($sFilename) - StringInStr($sFilename, "/", 0, -1))
                ExitLoop
            EndIf
        Next

    Else
        $sFilename = StringRight($aCracked[6], StringLen($aCracked[6]) - StringInStr($aCracked[6], "/", 0, -1))
    EndIf

    ; Close handles
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hHttpOpen)

    MsgBox(0, "", "URL=" & @CRLF & $array1[$i] & @CRLF & @CRLF & "Filename=" & @CRLF & $sFilename)

Next

 

Edited by KaFu

de.png OS: Win10-22H2 - 64bit - German, AutoIt Version: 3.3.16.1, AutoIt Editor: SciTE, Website: https://funk.eu
prog_icon_amt.png AMT - Auto-Movie-Thumbnailer (2024-Oct-13) prog_icon_bic.png BIC - Batch-Image-Cropper (2023-Apr-01) prog_icon_cop.png COP - Color Picker (2009-May-21)
prog_icon_dcs.png DCS - Dynamic Cursor Selector (2024-Oct-13) prog_icon_hmw.png HMW - Hide my Windows (2024-Oct-19) prog_icon_hrc.png HRC - HotKey Resolution Changer (2012-May-16) 
prog_icon_icu.png ICU - Icon Configuration Utility (2018-Sep-16) prog_icon_smf.png SMF - Search my Files (2024-Oct-20) - THE file info and duplicates search tool prog_icon_ssd.png SSD - Set Sound Device (2017-Sep-16)

Link to comment
Share on other sites
youtuber Universalist

youtuber

Posted
  • Author
Posted

I wonder why the URLDecode function between these codes doesn't work? :(

#include "WinHttp.au3"
#include <array.au3>
Global $types = "(?=.exe|.zip|.gz|.au3)"
Global $array1[5]

$array1[0] = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe"
$array1[1] = "https://web.archive.org/web/20190819012616/https://fs17.letsupload.co/kax4/test.au3?download_token=d2ef31a5e00970c4e8f5471e51116e56b14a39e23358ec9c60416b385bf1e49b"
$array1[2] = "https://www.apache.org/dist/tomcat/tomcat-7/v7.0.96/src/apache-tomcat-7.0.96-src.tar.gz.sha512"
$array1[3] = "https://web.archive.org/web/20190819014417/https://dl1.cdn.filezilla-project.org/client/FileZilla_3.44.1_win64.zip?h=V5YGZjZN40UvKJZ7bqD-Ug&x=1566182547"
$array1[4] = "https://web.archive.org/web/20190819014833/https://download.cdn.dll-files.com/fe907d306a80a1687d48919c6f4587c4/autoitx3.zip?VGd4d0hsa1NiQzVDdzNOMWVqaEtZUT09"

For $i = 0 To UBound($array1) - 1

    $aCracked = _WinHttpCrackUrl($array1[$i])
    ;_ArrayDisplay($aCracked)
    $ReadUrl = StringRegExp($array1[$i], "[\w -.%]+" & $types & "\.\w{2,3}", 3)

    $hHttpOpen = _WinHttpOpen()
    $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
    $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
    _WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
    _WinHttpSendRequest($hRequest)
    _WinHttpReceiveResponse($hRequest)

    $sDisp = _WinHttpQueryHeaders($hRequest)
    ; ConsoleWrite($sDisp & @CRLF)

    Local $sFilename
    If StringInStr($sDisp, 'Content-Disposition: attachment; filename=') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Content-Disposition: attachment; filename=') Then
                $sFilename = StringReplace($aDisp[$y], 'Content-Disposition: attachment; filename=', '')
                ExitLoop
            EndIf
        Next
    ElseIf StringInStr($sDisp, 'Location: ') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Location: ') Then
                $sFilename = StringReplace($aDisp[$y], 'Location: ', '')
                $sFilename = StringRight($sFilename, StringLen($sFilename) - StringInStr($sFilename, "/", 0, -1))
                ExitLoop
            EndIf
        Next

    Else
        $sFilename = StringRight($aCracked[6], StringLen($aCracked[6]) - StringInStr($aCracked[6], "/", 0, -1))
        ;ConsoleWrite($sFilename & @CRLF)
    EndIf
    ; Close handles
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hHttpOpen)

    If StringInStr($sFilename, "X-All") Then
        If IsArray($ReadUrl) Then
            ConsoleWrite(URLDecode($ReadUrl[0]) & @CRLF)
        EndIf
    Else
        ConsoleWrite(URLDecode($sFilename) & @CRLF)
    EndIf
Next

Func URLDecode($urlText)
    $urlText = StringReplace($urlText, "+", " ")
    Local $matches = StringRegExp($urlText, "\%([abcdefABCDEF0-9]{2})", 3)
    If Not @error Then
        For $match In $matches
            $urlText = StringReplace($urlText, "%" & $match, BinaryToString('0x' & $match))
        Next
    EndIf
    Return $urlText
EndFunc   ;==>URLDecode

 

Link to comment
Share on other sites
KaFu Universalist

KaFu

Posted
Posted 
#include "WinHttp.au3"
#include <array.au3>
Global $types = "(?=.exe|.zip|.gz|.au3)"
Global $array1[5]

$array1[0] = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe"
$array1[1] = "https://web.archive.org/web/20190819012616/https://fs17.letsupload.co/kax4/test.au3?download_token=d2ef31a5e00970c4e8f5471e51116e56b14a39e23358ec9c60416b385bf1e49b"
$array1[2] = "https://www.apache.org/dist/tomcat/tomcat-7/v7.0.96/src/apache-tomcat-7.0.96-src.tar.gz.sha512"
$array1[3] = "https://web.archive.org/web/20190819014417/https://dl1.cdn.filezilla-project.org/client/FileZilla_3.44.1_win64.zip?h=V5YGZjZN40UvKJZ7bqD-Ug&x=1566182547"
$array1[4] = "https://web.archive.org/web/20190819014833/https://download.cdn.dll-files.com/fe907d306a80a1687d48919c6f4587c4/autoitx3.zip?VGd4d0hsa1NiQzVDdzNOMWVqaEtZUT09"

For $i = 0 To UBound($array1) - 1
    ConsoleWrite($i & " URL= " & $array1[$i] & @CRLF)

    $aCracked = _WinHttpCrackUrl($array1[$i], $ICU_DECODE)
    ; Global Const $ICU_DECODE = 0x10000000
    ; Global Const $ICU_ESCAPE = 0x80000000

    ;_ArrayDisplay($aCracked)

    $ReadUrl = StringRegExp($array1[$i], "[\w -.%]+" & $types & "\.\w{2,3}", 3)
    ;_ArrayDisplay($ReadUrl)
    ; ConsoleWrite("$ReadUrl= " & $ReadUrl & @CRLF)

    $hHttpOpen = _WinHttpOpen()
    $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
    $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
    _WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
    _WinHttpSendRequest($hRequest)
    _WinHttpReceiveResponse($hRequest)

    $sDisp = _WinHttpQueryHeaders($hRequest)
    ; MsgBox(0,"",$sDisp)

    Local $sFilename
    If StringInStr($sDisp, 'Content-Disposition: attachment; filename=') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Content-Disposition: attachment; filename=') Then
                $sFilename = StringReplace($aDisp[$y], 'Content-Disposition: attachment; filename=', '')
                ExitLoop
            EndIf
        Next
    ElseIf StringInStr($sDisp, @CRLF & 'Location: ') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Location: ') Then
                $sFilename = StringReplace($aDisp[$y], 'Location: ', '')
                $sFilename = StringRight($sFilename, StringLen($sFilename) - StringInStr($sFilename, "/", 0, -1))
                ExitLoop
            EndIf
        Next

    Else
        $sFilename = StringRight($aCracked[6], StringLen($aCracked[6]) - StringInStr($aCracked[6], "/", 0, -1))
        ;ConsoleWrite($sFilename & @CRLF)
    EndIf
    ; Close handles
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hHttpOpen)

    ConsoleWrite($sFilename & @CRLF)
    #cs
        ConsoleWrite(URLDecode($sFilename) & @CRLF)
        ConsoleWrite(URLDecode(URLDecode($sFilename)) & @CRLF)
    #ce

    #cs
        If StringInStr($sFilename, "X-All") Then
        If IsArray($ReadUrl) Then
        ConsoleWrite(URLDecode($ReadUrl[0]) & @CRLF)
        ; ConsoleWrite(_URIDecode($ReadUrl[0]) & @CRLF)
        EndIf
        Else
        ConsoleWrite(URLDecode($sFilename) & @CRLF)
        ; ConsoleWrite(_URIDecode($sFilename) & @CRLF)
        EndIf
    #ce

    ConsoleWrite(@CRLF)

Next

_WinHttpCrackUrl > $iFlag - [optional] Flag that control the operation. Default is $ICU_ESCAPE

Change to $ICU_DECODE and the function takes care of decoding, otherwise it seems to encode the URL another time, so double-decoding leads to the desired result.

 

Was wondering about the "X-All" test, and I think X-Location: All has nothing to do with filename (guessed, further research needed), so adding a  @crlf to the filename test should take care of that too:

StringInStr($sDisp, @CRLF & 'Location: ')

de.png OS: Win10-22H2 - 64bit - German, AutoIt Version: 3.3.16.1, AutoIt Editor: SciTE, Website: https://funk.eu
prog_icon_amt.png AMT - Auto-Movie-Thumbnailer (2024-Oct-13) prog_icon_bic.png BIC - Batch-Image-Cropper (2023-Apr-01) prog_icon_cop.png COP - Color Picker (2009-May-21)
prog_icon_dcs.png DCS - Dynamic Cursor Selector (2024-Oct-13) prog_icon_hmw.png HMW - Hide my Windows (2024-Oct-19) prog_icon_hrc.png HRC - HotKey Resolution Changer (2012-May-16) 
prog_icon_icu.png ICU - Icon Configuration Utility (2018-Sep-16) prog_icon_smf.png SMF - Search my Files (2024-Oct-20) - THE file info and duplicates search tool prog_icon_ssd.png SSD - Set Sound Device (2017-Sep-16)

Link to comment
Share on other sites
youtuber Universalist

youtuber

Posted
  • Author
Posted (edited)

@KaFu If you have a download address like this one. 

https://web.archive.org/web/20190822040442/https://s282myt.storage.yandex.net/rdisk/2f60cabf31da26853111ecea602fab20b821cb8a46adee8ec053432d270b9179/5d5e4c62/gaG5jIPvspHWIXtY9Y9OLiQuoHsMiJSa1hDrtHd0afpQv8cnXgV2Ml0NHZ8PQRPLTEXzwtAKdmbHWSNZ4jc88g==?uid=0&filename=Test%20MsgBox.au3&disposition=attachment&hash=rul7sHfLjBcpP9UXDrB6I1xF%2BE7Oq5DM00C8S7gfNyQG6fl4MOf1bbmvRKZ5L0l4q/J6bpmRyOJonT3VoXnDag%3D%3D&limit=0&content_type=text%2Fplain&owner_uid=504872142&fsize=24&hid=3a4dfb00b6cdf0a59a7e58d84d380506&media_type=development&tknv=v2&rtoken=R7luhA5hrKUM&force_default=no&ycrid=na-de0032a7af481b74b2e26e3ec1fc7859-downloader8f&ts=590b021025c80&s=7acc9ef73b7b94b866b1c0d8c9e3d9d66d615858aafc6be4a2b5fabc95473a3f&pb=U2FsdGVkX1-bjAQlZgAx-SYi16bt0E9HuNEt_aXyM9nXXCbZy2FC5bQncI4MIEUGB5bNaHGJT91QrLlgRMkTSR9OPN0Wv2og0TyKDjHguCg

The answer I get in Header is this: Content-Disposition: attachment; filename*=UTF-8''Test%20MsgBox.au3
My attempt to do so failed have any suggestions for downloading it? 

ElseIf StringInStr($sDisp, 'Content-Disposition: attachment; filename*=UTF-8') Then
$aDisp = StringSplit($sDisp, @CRLF)
For $y = 1 To $aDisp[0]
If StringInStr($aDisp[$y], 'Content-Disposition: attachment; filename*=UTF-8') Then
$sFilename = StringReplace($aDisp[$y], 'Content-Disposition: attachment; filename*=UTF-8', '')
ExitLoop
EndIf
Next

;Console output: gaG5jIPvspHWIXtY9Y9OLiQuoHsMiJSa1hDrtHd0afpQv8cnXgV2Ml0NHZ8PQRPLTEXzwtAKdmbHWSNZ4jc88g==

 

Edited by youtuber
Link to comment
Share on other sites
KaFu Universalist

KaFu

Posted
Posted

It's really a case-by-case manual reconstruction of how a browser handles the http responses :)... 

#include "WinHttp.au3"
#include <array.au3>

Global $array1[6]

$array1[0] = "https://vorboss.dl.sourceforge.net/project/orwelldevcpp/Setup%20Releases/Dev-Cpp%205.11%20TDM-GCC%204.9.2%20Setup.exe"
$array1[1] = "https://web.archive.org/web/20190819012616/https://fs17.letsupload.co/kax4/test.au3?download_token=d2ef31a5e00970c4e8f5471e51116e56b14a39e23358ec9c60416b385bf1e49b"
$array1[2] = "https://www.apache.org/dist/tomcat/tomcat-7/v7.0.96/src/apache-tomcat-7.0.96-src.tar.gz.sha512"
$array1[3] = "https://web.archive.org/web/20190819014417/https://dl1.cdn.filezilla-project.org/client/FileZilla_3.44.1_win64.zip?h=V5YGZjZN40UvKJZ7bqD-Ug&x=1566182547"
$array1[4] = "https://web.archive.org/web/20190819014833/https://download.cdn.dll-files.com/fe907d306a80a1687d48919c6f4587c4/autoitx3.zip?VGd4d0hsa1NiQzVDdzNOMWVqaEtZUT09"
$array1[5] = "https://web.archive.org/web/20190822040442if_/https://s282myt.storage.yandex.net/rdisk/2f60cabf31da26853111ecea602fab20b821cb8a46adee8ec053432d270b9179/5d5e4c62/gaG5jIPvspHWIXtY9Y9OLiQuoHsMiJSa1hDrtHd0afpQv8cnXgV2Ml0NHZ8PQRPLTEXzwtAKdmbHWSNZ4jc88g==?uid=0&filename=Test%20MsgBox.au3&disposition=attachment&hash=rul7sHfLjBcpP9UXDrB6I1xF%2BE7Oq5DM00C8S7gfNyQG6fl4MOf1bbmvRKZ5L0l4q/J6bpmRyOJonT3VoXnDag%3D%3D&limit=0&content_type=text%2Fplain&owner_uid=504872142&fsize=24&hid=3a4dfb00b6cdf0a59a7e58d84d380506&media_type=development&tknv=v2&rtoken=R7luhA5hrKUM&force_default=no&ycrid=na-de0032a7af481b74b2e26e3ec1fc7859-downloader8f&ts=590b021025c80&s=7acc9ef73b7b94b866b1c0d8c9e3d9d66d615858aafc6be4a2b5fabc95473a3f&pb=U2FsdGVkX1-bjAQlZgAx-SYi16bt0E9HuNEt_aXyM9nXXCbZy2FC5bQncI4MIEUGB5bNaHGJT91QrLlgRMkTSR9OPN0Wv2og0TyKDjHguCg"

For $i = 0 To UBound($array1) - 1
    ConsoleWrite($i & " URL= " & $array1[$i] & @CRLF)

    $aCracked = _WinHttpCrackUrl($array1[$i], $ICU_DECODE)
    ; Global Const $ICU_DECODE = 0x10000000
    ; Global Const $ICU_ESCAPE = 0x80000000
    ;_ArrayDisplay($aCracked)

    $hHttpOpen = _WinHttpOpen()
    $hConnect = _WinHttpConnect($hHttpOpen, $aCracked[2], $aCracked[3])
    $hRequest = _WinHttpOpenRequest($hConnect, "GET", $aCracked[6] & $aCracked[7], Default, Default, Default, $WINHTTP_FLAG_SECURE)
    _WinHttpAddRequestHeaders($hRequest, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0")
    _WinHttpSendRequest($hRequest)
    _WinHttpReceiveResponse($hRequest)

    $sDisp = _WinHttpQueryHeaders($hRequest)
    ; ClipPut($sDisp)
    ; MsgBox(0, "", $sDisp)


    Local $sFilename
    If StringInStr($sDisp, 'Content-Disposition: attachment; filename') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Content-Disposition: attachment; filename') Then
                If StringInStr($aDisp[$y], "*=UTF-8'") Then
                    $sFilename = URLDecode(StringReplace($aDisp[$y], "Content-Disposition: attachment; filename*=UTF-8''", ''))
                    ExitLoop
                Else
                    $sFilename = StringReplace($aDisp[$y], 'Content-Disposition: attachment; filename=', '')
                    ExitLoop
                EndIf
            EndIf
        Next
    ElseIf StringInStr($sDisp, @CRLF & 'Location: ') Then
        $aDisp = StringSplit($sDisp, @CRLF)
        For $y = 1 To $aDisp[0]
            If StringInStr($aDisp[$y], 'Location: ') Then
                $sFilename = StringReplace($aDisp[$y], 'Location: ', '')
                $sFilename = StringRight($sFilename, StringLen($sFilename) - StringInStr($sFilename, "/", 0, -1))
                ExitLoop
            EndIf
        Next

    Else
        $sFilename = StringRight($aCracked[6], StringLen($aCracked[6]) - StringInStr($aCracked[6], "/", 0, -1))
    EndIf
    ; Close handles
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hHttpOpen)

    ConsoleWrite($sFilename & @CRLF)

    ConsoleWrite(@CRLF)

Next



Func URLDecode($urlText)
    $urlText = StringReplace($urlText, "+", " ")
    Local $matches = StringRegExp($urlText, "\%([abcdefABCDEF0-9]{2})", 3)
    If Not @error Then
        For $match In $matches
            $urlText = StringReplace($urlText, "%" & $match, BinaryToString('0x' & $match))
        Next
    EndIf
    Return $urlText
EndFunc   ;==>URLDecode

 

de.png OS: Win10-22H2 - 64bit - German, AutoIt Version: 3.3.16.1, AutoIt Editor: SciTE, Website: https://funk.eu
prog_icon_amt.png AMT - Auto-Movie-Thumbnailer (2024-Oct-13) prog_icon_bic.png BIC - Batch-Image-Cropper (2023-Apr-01) prog_icon_cop.png COP - Color Picker (2009-May-21)
prog_icon_dcs.png DCS - Dynamic Cursor Selector (2024-Oct-13) prog_icon_hmw.png HMW - Hide my Windows (2024-Oct-19) prog_icon_hrc.png HRC - HotKey Resolution Changer (2012-May-16) 
prog_icon_icu.png ICU - Icon Configuration Utility (2018-Sep-16) prog_icon_smf.png SMF - Search my Files (2024-Oct-20) - THE file info and duplicates search tool prog_icon_ssd.png SSD - Set Sound Device (2017-Sep-16)

Link to comment
Share on other sites
Nine Universalist

Nine

Posted
Posted

Maybe use IE instead as it will provide you with the right file name in the save as window ?

“They did not know it was impossible, so they did it” ― Mark Twain

Spoiler

Block all input without UAC

Save/Retrieve Images to/from Text

 Monitor Management (VCP commands)

Tool to search in text (au3) files

Date Range Picker

 Virtual Desktop Manager

Sudoku Game 2020

Overlapped Named Pipe IPC

 HotString 2.0 - Hot keys with string

x64 Bitwise Operations

Multi-keyboards HotKeySet

 Recursive Array Display

Fast and simple WCD IPC

Multiple Folders Selector

 Printer Manager

GIF Animation (cached)

 

Screen Scraping

 

 Multi-Threading Made Easy

 

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...