howto save a list of URLs to their *DEFAULT* file names?

[rudi]

Hi.

Eric Ligman posted tens of free Microsoft Books again:

Article

A list of download URLs to get all the books is available here I saved that one (holding the URLs only) to "c:\temp\MS-free-books.txt"

Some of the listed download URLs:

http://ligman.me/1IW1oab
http://ligman.me/1IW1s9S
http://ligman.me/1JIhgjA
http://ligman.me/1NDTZR4
http://ligman.me/1H4VXHT
[snip]

When I open these URLs one by one in a browser, I receive a "default name", mentioned in the "Save to..." box automatically. Howto use this "default name", when downloading the stuff using autoit? inetget() expects the "full-path-file-name" for the URL to be downloaded to. So this code ends up with useless filenames without extensions:

#include <array.au3>
#include <file.au3>

$BookList="c:\temp\MS-free-books.txt"
Dim $aBookList
_FileReadToArray($BookList,$aBookList)

_ArrayDisplay($aBookList)


$ToFolder="C:\temp\MicrosoftFreeBooks"
DirCreate($ToFolder)


for $i = 1 to $aBookList[0]
    InetGet($aBookList[$i],$ToFolder & "\" & StringTrimLeft($aBookList[$i],StringInStr($aBookList[$i],"/",0,-1)),0,1)
Next

 

Maybe a different function (that I miss) is what I need?

 

 

Any suggestions appreciated. Regards, Rudi.

Edited July 13, 2015 by rudi
typo in post topic

[guinness]

Nice link. Thanks

[rudi]

Nice link. Thanks

Nice, very helpful reply, thanks!

 

Now I really know how to solve my problem: Howto save all these files to their *DEFAULT* file names "offered" by the http server.

(Wasn't it *YOU* asking me, if I *DID* read some posts?)

 

Regards, Rudi.

[guinness]

Not me I'm afraid. Maybe another @guinness

[rudi]

Hi again.

 

This and that one aren't your posts?

Anyway: Do you have perhaps any suggestion, how to download all the files in the list i've linked in the OP to their *DEFAULT* file names?

 

Regards, Rudi.

[guinness]

My bad. I read something totally different as the sentences you wrote didn't make much sense to me. Sorry about that.

Yes that was me. I read your post, as I said thanks for the helpful link (which was actually helpful). Of course I could help you, but I feel the site you link to, wouldn't appreciate this form of mass downloading you're proposing.

[rudi]

Perhaps you could drop me a PM? I just need to know how to investigate (?) the default file names, that common download URLs offer, when clicking them "save as..."

And if I got he OP of this list, it's just changing too frequently, to offer a "all in one ZIP file". There are other tools that basically would also bulk download all in one, I'd just like to know how to do the trick "get the default file names" using autoit

 

Tx. Rudi.

[trancexx]

There are many ways for doing what you want. However they all have two things in common - you have to parse HTTP header to find Content-Disposition for files served as attachments or parse the final url in all others cases.
For example, WinHttp.au3 (find the thread in examples) can do both easily (of course that you can use any other available UDF):

expandcollapse popup

#include "WinHttp.au3"

$sUrl = "http://ligman.me/1IW1oab"

$sFile = FileNameFromWeb($sUrl)
ConsoleWrite($sFile & @CRLF)
MsgBox(4096, "File Name", $sFile)

Func FileNameFromWeb($sUrl)
    If StringLeft($sUrl, 7) <> "http://" And StringLeft($sUrl, 8) <> "https://" Then $sUrl = "http://" & $sUrl ; additional effort to make valid url



    ; Break the url into peaces

    Local $aURL = _WinHttpCrackUrl($sUrl)
    If @error Then Return SetError(1, 0, "") ; invalid url



    Local $hOpen = _WinHttpOpen()
    Local $hConnect = _WinHttpConnect($hOpen, $aURL[0] & "://" & $aURL[2])
    Local $hRequest

    If $aURL[1] = $INTERNET_SCHEME_HTTP Then
        $hRequest = _WinHttpSimpleSendRequest($hConnect, "HEAD", $aURL[6] & $aURL[7])
    ElseIf $aURL[1] = $INTERNET_SCHEME_HTTPS Then
        $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "HEAD", $aURL[6] & $aURL[7])
    EndIf



    ; Extract filename attribute
    Local $sFileName = __WinHttpAttribVal(_WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_DISPOSITION), "filename")
    ; ...and in case of any failure use the final url

    If Not $sFileName Then $sFileName = StringRegExpReplace(_WinHttpQueryOption($hRequest, $WINHTTP_OPTION_URL), ".*/", "")

    ; Close handles
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)

    Return $sFileName
EndFunc

 

Edited July 14, 2015 by trancexx
Changed to more complete function