Sequential downloading

[kurbads]

In sequential mode multipart RAR archives are downloaded starting with the *.r00 and not *.rar file. I understand it is alphabetical order. But what a heck?

If you deselect all other parts and download the *.rar first, VLC sometimes is able to play incomplete download. But not always. Can it be fixed?

[ciaobaby]

Can it be fixed?

Yes, ... ... By whomever created the torrent in the first place turning off "Maintain File Order" when they create the metadata info of the payload.

[kurbads]

For sequential download in case of multipart RAR archives, it does not require sequential download of individual parts. They are small anyway. It just needs normal download of individual parts one after another is completed in alphabetical order except the part with extension *.rar which comes first. Besides I do not think you would be able to play incomplete part of multipart RAR archive no matter how sequential it was downloaded.

If I was a programmer of qBittorrent, I would just add a subroutine with IF the files in torrent end with *.rar.

I understand there is incredible complexity behind sequential download feature. I congratulate the author about cracking it.

Though the multiple part RAR archive sequential download requires much lesser complexity.

Downloading each part in normal mode one file after another in case of multipart RAR archives would also circumvent sudden and sharp slowdown when sequential mode is selected.

To summarise for sequential download of multipart RAR archives:
1) fix the sorting (download *.rar part first, then the r00, r01, etc. parts);
2) disable sequential download within individual parts;
3) add sequential download of whole parts (the next part should download only after the previous one has been downloaded).

If possible unrar the file during download. It would be so much nicer to have unrared file with ability to seed as if it was rared at the same time. Multipart rar files have zero compression so it should not be hard to implement that by adding some rarinf or rardat file to the downloaded torrent and then reading the configuration from there while seeding.

[ciaobaby]

My sincere wish is that people who use the BitTorrent protocol for file transfer will someday understand that creating a torrent of 'split' archives is a futile exercise. The BitTorrent protocol DOES the 'splitting' of the payload into smaller 'chunks' already.

Split archives are fine for HTTP, FTP or SCP, but for BitTorrent ... It simply makes data transfer more inefficient than "sequential downloading" or "maintain file order" does.

It wastes huge amounts of disc space unless the individual split parts are the exact same size of, or an exact multiple of, a drive 'cluster'. With a single monolithic archive only ONE extra cluster in total is ever partially used, with multiple parts there could/will be one extra partially used cluster for EVERY SINGLE 'PIECE' the archive is split into.

SO


DON'T DO IT, .... There is no need for it, there is NO advantage, there is NO benefit to anyone.

[Nemo]

Just an addon; Sequential downloading is also bad on its own.

https://wiki.vuze.com/w/Sequential_downloading_is_bad
http://thoseonboard.com/2012/05/02/people-are-stupid-for-wanting-sequential-downloading-of-bittorrent-files-then-fix-it-einstein/

I would remove this feature completely if possible, it does more harm than good. But since this is a feature of qBittorrent thats not possible. I would advice not to use this way of downloading as a standard with every torrent you download.

[kurbads]

With an 16Gb movie it takes about 3 hours to download. If the movie is over 3 hours long with sequential download I can start watching straight away. It is brilliant feature. I wish it was possible to seed even before the file is being created e.g. during broadcast. Nobody is talking about doing sequential download for every torrent you download. Unfortunately my favourite tracker uses multipart archives as a must. Your advises are for seeders. My request is a leacher request.

[ciaobaby]

Unfortunately my favourite tracker uses multipart archives as a must.

Some people should NEVER be allowed to run trackers!

[sledgehammer_999]

Unfortunately my favourite tracker uses multipart archives as a must.

Some people should NEVER be allowed to run trackers!

Maybe splitting it into parts is a countermeasure against sequential downloading. It forces everyone to download the file in its entirety before watching it. And since sequential downloading is useless in this case they also have a healthier swarm.

**This is a theory of mine. I've never talked with anyone doing this practice.

[ciaobaby]

Maybe splitting it into parts is a countermeasure against sequential downloading. It forces everyone to download the file in its entirety before watching it. And since sequential downloading is useless in this case they also have a healthier swarm.

Using a monolithic archive would have the same effect, as they cannot be extracted until the entire archive is downloaded.

I think it's more likely that they do not understand how BitTorrent actually works and they assume that it is like HTTP or FTP downloading where a monolithic file download has to start from scratch if the connection is interrupted. Or maybe they think it prevents them being found complicit in copyright theft.

[zeomal]

@ciaobaby, this might be reason for splitting the file into several parts.

Most often, files are require to be split into archives by whichever torrent site it is being uploaded to. For example, many private trackers require that the torrent data is split into RAR or ZIP archives (the purpose probably is to help leechers, well, leech - as in the case where you do not want to download a particularly large file, but still want to increase your share ratio to keep it above a certain minimum value, as required on most torrent files). Most users of these private sites just re-upload the entire torrent to a public torrent site without bothering to unRAR or unZIP the files.

In addition, as per whatever I've read about the "scene", it is also required for whatever is uploaded to be accepted as legitimate. Read under Packaging in this Wikipedia article. Also, refer to the following for more information on the same here. This is possibly what reaches the private sites, and well, the hierarchy goes on.

[ciaobaby]

Most often, files are require to be split into archives by whichever torrent site it is being uploaded to. For example, many private trackers require that the torrent data is split into RAR or ZIP archives (the purpose probably is to help leechers, well, leech - as in the case where you do not want to download a particularly large file, but still want to increase your share ratio to keep it above a certain minimum value, as required on most torrent files). Most users of these private sites just re-upload the entire torrent to a public torrent site without bothering to unRAR or unZIP the files.

Oh man!! If that's the best excuse they can come up with for an explanation of their crass stupidity, they should give in and go work for a train company as an "Excuse writer" for when the train is delayed.

ANY 'private' tracker that imposes ridiculous "rules" that you have to comply with, is really not worth your patronage of them.

[Nemo]

Both are right.

Why are releases compressed into multiple RAR archive parts?

The simple answer, is that this is what the scene rules specify. These rules make it easier to manage files for members of the scene. When transferring data, corruption can occur at any time, so when a file becomes corrupted during transit, for instance, from a topsite to a users computer, it has to be re-downloaded. For very large files, this results in a lot of wasted bandwidth and time. To avoid this, the file is split into several smaller archived parts, which are downloaded individually. If one of these parts is found to be corrupted, all a user must do is re-download that small part.

Taken from Wiki:
Release groups -> Topsites (private ones) -> Couriers -> Sites -> Leechers.

http://scenelingo.wordpress.com/2008/08/16/the-rar-argument
http://en.wikipedia.org/wiki/Warez_scene

[sledgehammer_999]

And as ciaobaby has already mentioned, bittorrent has made that argument invalid and obsolete. It(bittorrent) guarantees that it will detect corruption as soon as it happens and also it guarantees that you won't need to redownload the whole large file.

[Nemo]

Indeed.

[Goblin]

One reason why files are archived in multiple parts is because they are transferred like that between scene sites. FTP with checksum files is the preferred (only) method. Once releases leak out of the scene, that structure is maintained a lot of places, because the one thing worse than having multiple archives is having an unpacked and potentially incomplete/tampered with release.

Another reason is that not everyone downloads to their home box. Most people who have any kind of say in what happens on private trackers use seedboxes, and they transfer the files to their home boxes via FTP or HTTP.