TheSaint

You can also use _FileReadToArray instead, and see the array number as line number.

Yep I came across that a week or so ago. I think I may have even mentioned that in a previous post. Alas it is only 64 Bit though. Yes I did. Anyway I am quite happy using what I am using, which is mostly Free Download Manager 5 at the moment, due to the speed benefit. I have tweaked my code enough, so that it works pretty well. Now that I seem to have finished the tweaking and testing, I will probably rotate between aria2 and FDM5, depending on the overall size of the game files to download for a game.

No longer the case, as I couldn't resist completing the code today, and trying it out successfully with several game downloads. And after tweaking it a bit it works well. One of the files was a 9GB Linux version game file, which like the rest downloaded in a satisfying portion of time. So now my program offers 4 different methods to download your purchases from GOG. 1. The original single thread gogcli.exe. 2. Single thread curl.exe. 3. Multi-thread aria2.exe with curl.exe. 4. Multi-thread fdm.exe with curl.exe.

Well, I've not done that yet, due to wanting to import the code to my GOGcli GUI program, which I have now done and tweaked and tested a lot, and it works well. That is, the aria2 with curl portion of my code, and it took some tweaking to get it exactly how I wanted ... progress feedback and all. At this stage, I have only done the preliminary work for Free Download Manager support, so not available for use yet. In a way, I felt I no longer need it, so I just did some elements as a start in case it is ever needed. I no longer feel quite that way, as I have come across a limitation with aria2 ... the numbers of threads you can use with GOG, which appears to be a maximum of five. Five threads will give me somewhere around 5 Megabytes a second, but usually less on average ... some days and times are clearly better than others. On the other hand, I could get a fair bit more with Free Download Manager 5, which must be doing something tricky to get around that seeming 5 thread limitation. I know I can bump the threads in FDM5 up to about 11 and get close to 7 Megabytes a second at times. The aria2 program reports on how many threads is in use, and I have never seen it above 5, and the speed indicates that too. So it is very clear when using FDM5, that it can be faster. Here's the latest screenshot of the downloading window of my GOGcli GUI program, after I reworked it a bit to fit some new or changed elements in. I've grabbed over 74 game updates in the last few days, most of them either demos or prologues ... 118 GB worth. So finally I have made a start and dent on all the updates I need to get at GOG ... more than 100 still to go, of mostly full games ... so a lot of gigabytes. Anyway ... I will eventually get back to tidying up that code example and providing here.

Well I can be like a dog with a bone sometimes. I worry at it until I succeed. I finally got aria2 to work, and yes it was Win 7 32 bit related. Another search on my socket error, eventually brought me to the following posted issue. https://github.com/aria2/aria2/issues/1801 So going by that I grabbed a static version of aria2, and tried that. That failed with a certificate error. So as I had also read how to provide the certificate on the command-line, I did that ... and whamo it worked. $params = $aria2 & ' -c -d "' & $downfold & '" -o "' & $file & '" -x ' & $threads & ' --ca-certificate="' & $cert & '" "' & $URL & '"' $pid = RunWait($params, "", @SW_SHOW) I'll provide the full script, once I have tidied a few things up, but could not contain my excitement right now, so had to share. Anyway, once again a BIG THANKS to my buddy @TheDcoder for his original suggestion and command-line and helpful comments along the way at various stages. We got there in the end ... what a journey.

As I was waking today, and cogitating over things, I suddenly remembered that FDM5, like many downloaders, downloads to a temporary file first until the download has finished, after which I presume it just renames the temp file to the desired file name. So it would appear I can just rely on file exists and byte size to determine when a download has completed. Anyway, that is what I am testing right now, among other things. EDIT While that works, I am still needing to tweak things due to timing issues etc. EDIT 2 Okay, I appear to have that sussed now. Time to investigate other options, etc. 1. See if I can determine via code, what download folder FDM5 is currently set to use. ONGOING - Seems it is set on a file by file basis, with there being a recorded history and the last folder used is the current one, and from what I can tell is only listed in a sqlite database entry, which one might be able to modify to get a changed result. 2. Download latest aria2 and see if they have fixed anything for my use. DONE - Already using the latest version from November 2023. 3. Look again, for another command-line multi-thread downloader program. DISCOVERED that wget2 now exists, and is multi-threaded, but only 64 bit. I was also reminded of httrack, which I might take another look at now.

Well I never got to the point of doing that this time around. And as I got the curl side of things working anyway, probably not an issue anyway. Though maybe I will try it to see if it makes a difference to aria2 ... probably not.

But on the good result side, I can now successfully pass the URL to Free Download Manager 5 on the command-line, even if it is already running. Alas though, I cannot specify the output folder via command-line. So now I need to look at how I can leverage this to my best advantage. EDIT I just discovered that every download with FDM5 using this method (at least), gets a different PID. So now it looks like we might be starting to cook with gas, as that gives monitoring options. But alas, those PID are short lived. Even using RunWait does not wait until a download completes. So now I am stuck at needing some way to know when a download has completed. I could do the 'FlushFileBuffers' thing, but that seems to me it would have an element of presume. I need to know when FDM5 has finished with a download, so it can be relocated and then validated.

Still no luck with aria2. I keep getting the following error message. The command I am using is the following. $pid = Run(@ComSpec & " /k " & 'aria2c.exe -c -d "' & $downfold & '" -o "' & $file & '" -x 4 "' & $URL & '"') And so we appear to be back at why I could never get aria2 to work for me ... not on this PC anyway.

Okay I had to give up on BATCH files and just use AutoIt, and now I am getting good results with the following. #include <Constants.au3> #include <WindowsConstants.au3> #include <Misc.au3> #include <File.au3> Global $curl, $data, $downfold, $download, $file, $link, $params, $pid, $URL _Singleton("curl-get-thesaint", 0) $curl = @ScriptDir & "\curl.exe" $downfold = @ScriptDir & "\Files" $file = "setup_underground_blossom_demo_1.0_(67138).exe" $download = $downfold & "\" & $file FileChangeDir(@ScriptDir) $URL = "/downloads/underground_blossom_demo/en1installer0" $link = "https://www.gog.com" & $URL $params = $curl & ' -L -s -w "%{url_effective}" -r 0-0 -o /dev/null --cookie gogcom_cookies.txt "' & $link & '"' $pid = Run($params, "", @SW_SHOW, $STDOUT_CHILD) ProcessWaitClose($pid) $data = StdoutRead($pid) $data = StringReplace($data, "%28", "(") $URL = StringReplace($data, "%29", ")") MsgBox(262144 + 64, "CURL Result", $URL) $params = $curl & ' -# -o "' & $download & '" "' & $URL & '"' $pid = Run($params, "", @SW_SHOW) Exit

No luck with aria2, and no luck with FDM5. Not even any luck with curl as just a single thread, as it complains about a missing port number. WTF Testing with a BAT file only at this point. The following is my attempt with curl as the downloader, and where I get that port number error. @echo off "D:\Projects\GOG-CLI\Curl\curl.exe" -L -s -w '%%{url_effective}' -r 0-0 -o /dev/null --cookie gogcom_cookies.txt "https://www.gog.com/downloads/underground_blossom_demo/en1installer0" > Address.txt set /p url=<Address.txt echo %url% D:\Projects\GOG-CLI\Curl\curl.exe -# -o "D:\Projects\GOG-CLI\Curl\Downloads\game1.exe" %url% pause cls exit I'd also tried piping the result as follows, without using a variable, but that gives the following error message. @echo off "D:\Projects\GOG-CLI\Curl\curl.exe" -L -s -w '%%{url_effective}' -r 0-0 -o /dev/null --cookie gogcom_cookies.txt "https://www.gog.com/downloads/underground_blossom_demo/en1installer0" | D:\Projects\GOG-CLI\Curl\curl.exe -# -o "D:\Projects\GOG-CLI\Curl\Downloads\game1.exe" pause cls exit

And so it seems. As I now recall all my experiments with the command-line you gave me. The following is what I have just tried, in a BAT file, and it returns an error. D:\Projects\GOG-CLI\Curl\curl.exe -L -s -w '%{url_effective}' -r 0-0 -o /dev/null --cookie gogcom_cookies.txt "https://www.gog.com/downloads/underground_blossom_demo/en1installer0" So it shows the following in the console window. And then and old memory surfaced about % being used in a BAT file. That sometimes you need to use %% instead of %. And bugger me, that worked. And so maybe now I can get aria2 to work ... if that was the issue.

To be honest bud, I don't recall that, possibly because I did not fully understand what was going on. I knew it was getting the URL and passing it to aria2, and that was about it. Okay, I am a bit clearer on things now, so I should hopefully be able to get that working for me. THANKS. And no I couldn't get it working, but my memories are all tied up in all the attempts at getting aria2 to work anyway. And when that failed still after many tries, I then thought about just retrieving the true URL, but couldn't get a result for that either ... or maybe that is where it failed with aria2 ... so maybe we have been here before, and so I won't get it working. P.S. By the way, in trying to get aria2 working I went and downloaded many prior versions of aria2 and tried them, hoping one would work. Basically, to get the growing size of a file being downloaded, so that we can show that in the progress bar, we need something to refresh the file regularly. If you don't refresh it, then it remains zero bytes until the download is finished, after which time the system updates things. Some time back, years now probably, I asked in AutoIt Help, and someone provided me with the 'FlushFileBuffers' code I am using. Some time later, more than a year I am guessing, you happened to see that code in one of my scripts and replied that there was a better way. In this latest case, i am monitoring more than one file as it downloads, so the folder content really, which is just limited to the parts of a file being downloaded. Currently for that to work properly, I need to refresh each part file as it downloads. So I am guessing there is probably a better way of just refreshing the folder content as a whole, rather than each individual file part? Of course. So in other words the same as downloading two similar files. So twice the impact on the drive. But in the bigger picture of things, that is not a lot really. Yes it would be better if I didn't have to do it, but the impact is not anywhere near enough for me to lose sleep over or worry terribly about how much quicker my drive will die. So I deem the impact as rather minor overall. I've always done a lot of downloading and deleting and moving files around etc, and most of it won't involve this sort of thing. It's not like I download many files every day from GOG. Not that I have actually implemented it yet, and may never do so, just solely been testing things. It might be a good temporary thing though, until my PC situation improves. But if I can get that URL via curl and pass that to Free Download Manager for instance, which is a multi-thread program, then have it successfully download my GOG game files, then that would be the better solution, until I am able to take advantage of aria2. The problem with FDM5 is that it has extremely limited command-line support ... just IN (and maybe OUT) as I recall. If I can control OUT, then I am cooking with gas, as that means I can automate file validating etc. EVen if I cannot control where OUT will be, I know where the default is currently set, and can ultilise that. The other significant benefit, will be not having to grab every URL one by one like I currently have to, from my GOG library web page for a game. So even just that last benefit will be reward enough.

I have probably been remiss in not mentioning the following help I got from my good buddy TheDcoder, and now stated in my first post in this topic. A BIG THANKS to @TheDcoder for the '-L -#' command-line switches, that enabled me to finally be able to download files from GOG using curl.exe. They provided curl with the much needed redirect command for a path like the following. https://www.gog.com/downloads/brok_the_investigator_prologue/en1installer0 P.S. I've also amended my script, by adding a thank you line to TheDcoder there, near the start. EDIT By the way bud, I seem to recall you mentioning in the past, that there is a better way to get the growing file size, that didn't use "FlushFileBuffers". Or were you just referring to the _WinAPI_FlushFileBuffers command in the Help file? ... which unfortunately doesn't have an example, but I imagine does the same basic thing as the DLL command I use. $handle = FileOpen($filepth, 0) DllCall("Kernel32.dll", "BOOLEAN", "FlushFileBuffers", "HANDLE", $handle) FileClose($handle) Also ... I never did work out how to get curl.exe to return a value for me, of the full download URL from the redirect, so I could use it with some other downloader, that would let me gain the benefit of multi-streaming. Though it has now just occurred to me, that I might just be able to replace aria2 in the command-line example you gave me, with another program (i.e. Free Download Manager 5). The issue I had with aria2, according to my investigations, is possibly to do with Win 7 and 32 bit, and many other folk have the same issue from what I have discovered. I will eventually of course, not be using Win 7, but it is just a convenience for me right now, until I get some space etc sorted.

Today I decided to flesh out my code some more, adding improvements. Part of that is now specifying how many threads (parts) you want to download of a file. Anyway, I have added a bunch of helpful things. Most of my recent tests have been using 6 threads, which has worked real well with a great improvement to download speed. If you have a manifest file, you can also copy the data for each file, one at a time, to the clipboard, and the required values will be extracted from that automatically. Here's some screenshots. Here's the updated script Download In Parts.au3 (1 prior download)