Moving Torrents or Downloads stump Upload Speed

Windows specific questions, problems.
Post Reply
northstar99

Moving Torrents or Downloads stump Upload Speed

Post by northstar99 »

When I am not downloading anything, my upload speed works as it should. However, if I download anything, especially at fast speeds, the upload seems to throttle. I have a 1 Gbps down and 50 Mbps up, so it's not like the upload is anything monstrous for the download to bottleneck it out. It does seem like if I keep the download to less than 10 MB/s, it won't harm the upload. But any higher than that, especially in the upwards range of 60MB/s, the upload will constantly bounce up and down from full to 0 until the download finishes.

I believe when I move torrents from one drive to another (long process for big torrents) that there's no problem if I move just one. But just now I started moving about 8 x 500+GB torrents from one drive to another and the upload flat-lined to 0 and has say at 0 all day.

I have about 1600 torrents in Qbitorrent right now. About 25 TB worth of files being seeded. I believe I've tweaked the settings to be as upload friendly as possible, such as setting disk cache to -1 (auto managed), higher buffer watermarks, no limit to number of seeders etc... What settings should I be focused on to help alleviate the upload issue when downloading or moving files?

If you're thinking it may be my hard drives which are bottle-necking, the current drive is an 8 drive RAID10. I think it had around a 1 GB/s sequential Read speed and something like 600-700 MB/s seq write speed. Whatever it is, it's much faster than my internet connection and way faster than my max upload rate. The drive I am currently moving the files to is a 16 x RAID10 set-up which got a smoking 6 GB/s read & 7GB/s write (makes no sense, right?) for smaller writes of 50 MB a piece and 3.4 GB/s read & 2.2 GB/s seq write for 1 GB pieces of data so I think if the drive is the culprit, this new one won't be. The random read and write is on a nice 450 MB/s rate also.

I get that during transferring of too many torrents at a time that it might cause Qbittorrent to halt the uploads until it's done since the drive is likely running at it's max speed, although my OS says only 110 MB/s so it really isn't. But even if moving mass files bottle-necks it, I would think a download of only 20 MB/s wouldn't kill my upload / cause it to jitter to 0 and back up to max repeatedly. Another interesting detail, now that some of the moving torrents have finished, my drive is actually reading at 500 MB/s unlike the ~110MB/s it was doing earlier today.

Is there some known issue with Qbittorrent where it can't handle doing many torrents at a time for moving files? And when downloading, it can't keep up with reads for upload? Or is there a setting I'm missing that could help prevent Qbit from prioritizing other tasks over upload? I realize most users normally only care about download but I really want to prioritize my upload since the speed is slow as it is.

Also, there is another drive that seeds some of the torrents and is not currently active doing anything so that's part of why I think it's the program and not the drive. Otherwise, you'd think those drives could be uploading right now since they aren't doing anything. And yes, there is always someone who wants data off those 10 TB drives. They are normally uploading when there isn't download/move activity in qbittorrent.
fusk

Re: Moving Torrents or Downloads stump Upload Speed

Post by fusk »

Rather certain what you're experiencing is an i/o issue. Simply put, when moving large torrents, you raid can't keep up with the large write while also serving many small random reads. If you look in task manager, is your disk activity saying 100% while transferring slower than potential max?

I get up to 700mb/s when transferring from temp drive to storage, but while doing so upload is hurt across the board if i have many active seeding torrents even if there's few leechers on each as they're all requesting random reads from disk if the data isn't in memory.
northstar99 wrote: Sat Nov 09, 2019 6:30 am The random read and write is on a nice 450 MB/s rate also.
At which scale and queue depth 16, 32, 64kb? Torrent traffic is different from disk transfers and is more i/o heavy because of many small random i/o's.
Post Reply