Resuming partial downloads

[B.K. Johnson]

I'm no expert on wget, but I gather from the man pages that using it with -c will enable continuation of a partial download.

Today I came across this article How to Use Zsync to Transfer Part of a File in Linux which seems to make things easier. Zsync is in PPM. I have installed it but I haven't tried to download with it yet. Others contemplating a large download may want to try and report.

[nic007]

It depends whether the server you are downloading from allows for resumes. So even if you configure your downloader to always resume downloads it will still depend upon the download site.

[bigpup]

The browsers I use seem to control downloading.
The process is built into the browser.

Example:
Pale Moon browser.
Has options for where to install a download.
Shows a download process indicator.
Tells you when the download completes.
I have seen it stop a download and give message why and ask if you want to resume download.

Pale Moon has a add-on that will provide more info and options for downloading stuff.
https://addons.palemoon.org/addon/downloadstatusbar-pm/

[B.K. Johnson]

I think that bigpup is correct, that the download is controlled by the browser.
The scenario I had in mind when posting the question was if say you had a power failure of some duration leaving an orphan. If you were using a browser, then you could bring up the browser's Download Manager and click the "reconnect" button*, the browser would query the server and realign the download and continue. In my experience, this sometimes fail and the user has no choice but to restart the download from the beginning. A real PIA for a large file, e.g. ISO. However, if using wget, you must use the -c argument after an orphan exists, else it too restarts at the beginning, automatically. Zsync, it seems will automatically ALWAYS pick up where the orphan ended. I am not going to deliberately abort a download to try it, but the next time I need to download a large file I'll try Zync. In recent downloads, I have found wget and wget -c are better than browser downloads, so that trial may be soon. Stay tuned!

*The way I operate does not allow me to always have the download history available.

[nic007]

Most servers use a timeout period. Many for example have a 24-hour period which means that from the start of your download, the download link will be "active" or "alive" for 24-hours (no matter what downloader you use). During this period you can stop and resume the specific download. After the active period, you can not resume where you have last stopped but the download will begin again from the start. Many servers do not allow resumes at all, if the download stops for any reason you will have to start from the beginning again. Trust me I do lots of big downloads and have learnt the hard way. The safest way to counter these servers that do not allow for resumes, is to use one of these online download sites which can provide an alternative download link. What happens in the latter case is that the original link is used to download the file "internally" to another server (this is normally quite a quick process). The other server then provides a new (alternative) download link for the file with resume capabilities for a specific time period. BTW - I use a download add-on for my browser called Downthemall! With this I can immediately see if my download can be resumed or not.

[B.K. Johnson]

@nic007
Thanks for the info re "active" or "alive" period and I'll try to remember to use the "mirrors" rather than the main server in the future. Funny, I have always opted to download from the main server.

[nic007]

@nic007
Thanks for the info re "active" or "alive" period and I'll try to remember to use the "mirrors" rather than the main server in the future. Funny, I have always opted to download from the main server.

I don't think the "mirrors" will work any different because it's just another "pathway" for the original download link (in other words the download link is the same). You need a completely new download link and the only way to achieve that is when the actual file is actually transferred/downloaded/copied to another server. What I'm saying here is of course only applicable to those downloads that do not have resume capabilities.

[B.K. Johnson]

So, nic007, if I understand correctly, ordinary folks don't know

when the actual file is actually transferred/downloaded/copied to another server.

or do we, can we?

[nic007]

So, nic007, if I understand correctly, ordinary folks don't know

when the actual file is actually transferred/downloaded/copied to another server.

or do we, can we?

Some online download sites can download/copy the file "internally" to their private servers first and then you download it from there. Generally you will provide the original download link, it then downloads/copies the actual file to their servers in quick time and provide a new link where you can download it from.

[Semme]

This sounds appropriate to add to the discussion >> What determines whether downloads are resumable or not?

As to how your download manager can tell, it sends an HTTP "HEAD" request. If the response code is 206 (partial content), then the http stream supports resumption.

Ref.

Example(s): HTTP Range Requests

==

How can I find out whether a server supports the Range header?

How can I test a server for byte range support?

Testing HTTP Status: 206 Partial Content and Range Requests

[B.K. Johnson]

nic007, I understood what you wrote the first time. It seems I did not make myself clear. What I was asking and it seems clear to me, now that I read the link from Semme, that when anyone goes to download from any HTTP site, he does not know beforehand and cannot determine before whether his/her download will be resumed. There are "things" at the server level that the down-loader is not privy to that may preclude resumption. We take our chances and take what we get. End of story!

For downloads via browser, it is the responsibility of the browser to sort out the important details that the server sends and act so that we the client can have resumption when needed. The link from Semme reads in part:

This matches the URLSessionDownloadTask behavior which states that a download can only be resumed if the server provides either the ETag or Last-Modified header (or both) in its response.

The latter is to ensure that there is no resumption when the file has been changed, thereby avoiding a download that is part old and part new.

[nic007]

Yes, the sweet and short is that some downloads are resumable (some only for a specific period) and some are not.You will only know wnen you start a download link if it is resumable or not. If you use a downloader with a GUI you will probably see if it is resume enabled as soon as you load the download link (resume button will be active). If the file is not resumable, that's it. I have given you another option of what you can do if the download is not resumable. As far as browsers are concerned - most modern browsers will support the resume function (if the download server allows it).