Author Topic: Torrent read/write nvme benchmark.  (Read 1364 times)

fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #15 on: July 09, 2019, 11:15:09 AM »
Another important thing to mention on why I don't experience bad torrent problems is I mostly use uTorrent (v3.5.5 now) -- it's much better at handling file reads/writes than libtorrent does in qBitTorrent.
I see, i knew there were some issues with libtorrent, afaik it was noticeable issue, but not a major one. When you say "much better" what are we talking here?
uTorrent has been saving whole pieces in its cache and writing them to disk in 1 go for over a decade. qBitTorrent still doesn't seem to do that -- it'll just write maybe 512 KB at once.

Hm, maybe i should test that. Would you mind sharing your settings? Not that i expect them to match my setup. But there's probably settings worded differently, or settings you must change that one might overlook. 3.5.5 is the latest version? For some reason they seem to hide that very well on their site, and i just remember in the past some people were religiously using previous version because of things.
Also, would you happen to know how rtorrent fares in all that, it's not build with libtorrent IIRC?

Edit: utorrent is a mess tbh, crapware and installs crapware without consent, ads. That's first impressions so not saying it won't turn out well. But having looked through basically all settings, it seems like it's going to be impossible to set up utorrent to replace qbit with the watch folder setup i have where torrents comes from autodl through dropbox. But will have to give it some more time and see if it's possible.

But yes, other than that I'm a casual user, as you like.

Just curious, mind sharing your settings and advanced?
« Last Edit: July 09, 2019, 03:45:06 PM by fusk »

Switeck

  • Forum addict
  • ****
  • Posts: 1518
  • Karma: +96/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #16 on: July 10, 2019, 06:15:11 AM »
Hm, maybe i should test that. Would you mind sharing your settings? Not that i expect them to match my setup. But there's probably settings worded differently, or settings you must change that one might overlook. 3.5.5 is the latest version? For some reason they seem to hide that very well on their site, and i just remember in the past some people were religiously using previous version because of things.
Also, would you happen to know how rtorrent fares in all that, it's not build with libtorrent IIRC?
Follow the guide here:
https://forum.utorrent.com/topic/60691-utorrent-v2x3x-best-practice-tips/
...and even grab the settings.dat for 1+ mbit/sec upload (you'll definitely want to change the speed settings on it tho!)
You'll need to check the advanced settings carefully for some of the ad settings added since the settings.dat was created.

And that is pretty close to my settings, although I create some 0-byte read-only files to partially interfere with the ads.

This guide is helpful in understanding uTorrent's advanced settings:
https://www.netcheif.com/Articles/uTorrent/html/AppendixA_02_12.html
Understanding them can even help in figuring out how qBitTorrent works and should work.

buratino

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #17 on: July 10, 2019, 08:55:21 AM »
Just curious, mind sharing your settings and advanced?
I'm using Server 2016, maybe that's something compared to Win10. uTorrent doesn't work for me on Server 2016 (tried on several 2016 machines) - it glitches very seriously (at least a year ago when I tested several 3.xx versions) when speeds go over about 100 mbit (freezes to a crawl and it can't actually reach even 20MB/s). So it's totally out of the question and uT is dead for me. 2.2.1 worked like a charm though. Maybe newer versions are not well meant or compatible with Server. But even with 2.2.1 I couldn't see anything higher than 96MB/s and as small as the difference is, I insisted to see 100+ MB/s, so qBit succeeded at that, and I can see in my router ~steady 900-940 mbps. Average connections with 20-30 torrents shows around 5-600 in the router but I've seen 1500+ on busy torrents.
Torrents download fast enough so there is not much simultaneous upload while they download though.
Whatever I do, TBH I couldn't find enough torrents or be fast enough to load very new torrents with very few seeders and thousands of leechers to be able to be among the top seeders. Normally seeders are enough to feed the leechers so I couldn't see very high uploads even after dload completes.

My settings include: no preallocate files; manual management mode; static non-UPNP incoming port; global connections 2000, 200 per torrent, 100 upload slots (50 per torrent); enabled DHT/PeX/local_discovery; no torrent queueing.
In Advanced settings all default except: 512MB cache, no OS cache; no resolve peer countries or host names; no recheck torrents on completion; async IO threads - 4; coalesce read/writes - yes; max number of half-open conn - 200; outgoing ports min/max - 0/0; upload slots - fixed. Other things I think defaults.
I tried much bigger cache (1.5GB and over) but I didn't notice any difference above 512MB.

Again, I can't tell what would happen with 200-1000+ torrents loaded, I'm not an uploader or some busy torrent-guy. I guess they'll also fall into the global limit of 2000 connections though, if not increased further.
Anyway, I hope you find your solution and if that's about buying a new faster SSD, so be it. I just shared my experience, in no way I say it's universal.

fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #18 on: July 13, 2019, 06:40:37 PM »
Follow the guide here:
https://forum.utorrent.com/topic/60691-utorrent-v2x3x-best-practice-tips/
...and even grab the settings.dat for 1+ mbit/sec upload (you'll definitely want to change the speed settings on it tho!)
You'll need to check the advanced settings carefully for some of the ad settings added since the settings.dat was created.

And that is pretty close to my settings, although I create some 0-byte read-only files to partially interfere with the ads.

This guide is helpful in understanding uTorrent's advanced settings:
https://www.netcheif.com/Articles/uTorrent/html/AppendixA_02_12.html
Understanding them can even help in figuring out how qBitTorrent works and should work.

I tried it, but the big problem was that i use watch folder together with autodl to keep things organized, this works perfectly in qbit. Utorrent can't be made to work with that. I found 3'th party tools to solve it, but they are very old, have not been updated for +5 years and does not appear to be working on newer os's. So i had to go a different route.


My settings include: no preallocate files; manual management mode; static non-UPNP incoming port; global connections 2000, 200 per torrent, 100 upload slots (50 per torrent); enabled DHT/PeX/local_discovery; no torrent queueing.
In Advanced settings all default except: 512MB cache, no OS cache; no resolve peer countries or host names; no recheck torrents on completion; async IO threads - 4; coalesce read/writes - yes; max number of half-open conn - 200; outgoing ports min/max - 0/0; upload slots - fixed. Other things I think defaults.
I tried much bigger cache (1.5GB and over) but I didn't notice any difference above 512MB.

Those seem reasonable.

fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #19 on: July 16, 2019, 12:05:32 AM »
Ok, i was not expecting this kinda difference, this is nuts. I mean, we're not just talking about a 10% improvement here from switching away from libtorrent. It's a lot more than that.
Average upload speed is like double, especially during download at higher rates.
« Last Edit: July 17, 2019, 01:22:10 AM by fusk »

buratino

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #20 on: July 16, 2019, 08:46:28 PM »
And what is this "away from libtorrent"? What is this program?
And this screenshot is not helpful, the sum of speeds is well over 1Gbit/s, do you have by chance 10Gbit internet?

fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #21 on: July 17, 2019, 01:08:02 AM »
Just noticed i didn't mention that. It's biglybt. Had my doubts, but it was open source so thought i'd give it a go. So far the only client i've found that can handle the multiple watch folder setup i have.
The difference, qbit using libtorrent 60 connections & 60 upload slots per torrent i get overloads on scratch disk using 3gb cache. With bigly and 150 connections & 150 upload slots per torrent everything is fine, no overloads using 600mb cache.

Picture wasn't meant to "help" anything as more time is needed to make any conclusion, it was only to show what speed now looked like. But i removed to avoid any confusion. When a week has passed i can compare it against other weeks, or months, that'll probably be better.
« Last Edit: July 17, 2019, 01:16:41 AM by fusk »

fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #22 on: July 19, 2019, 11:53:08 AM »
I gave up, biglybt worked. But wasn't allowed on all trackers. Tixati stores incomplete pieces on c: and has no manual cache settings. I admit defeat, and installed utorrent. And even tho it has no options to organize anything, i suppose i'll let anarchy rule until libtorrent 1.2.2.

Nemo

  • qBittorrent Forum
  • Administrator
  • Forum addict
  • *****
  • Posts: 1518
  • Karma: +94/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #23 on: July 19, 2019, 07:39:56 PM »
I have tried and used Biglybt too in the past for a while and I liked it. Its Vuze but without any crap build in and its being maintained. My speeds were quite good and stable too to be honest. Only downside is Java usage but most computers have minimal 8GB or more in 2019 so memory ain't a problem. But im wondering how much memory it would consume when there are 500+ torrents loaded into it for example.

I always say use whatever works the best for you ;).
Forum Rules and Guidelines

Forum Admin.
Dutch & Turkish Translator.




fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #24 on: July 20, 2019, 07:45:05 PM »
I have tried and used Biglybt too in the past for a while and I liked it. Its Vuze but without any crap build in and its being maintained. My speeds were quite good and stable too to be honest. Only downside is Java usage but most computers have minimal 8GB or more in 2019 so memory ain't a problem. But im wondering how much memory it would consume when there are 500+ torrents loaded into it for example.

I always say use whatever works the best for you ;).

Problem was that some private trackers didn't allow it, got contacted by staff that told me to switch and they wouldn't tell me why. So never knew what the issue was. But speeds were much better., but forced to switch clients.
Had a look at tixati, which also seems like a good client, but it has some weird defaults, like it stores incomplete pieces on c: and you can't adjust write cache. Ran out of ideas.

Now i am actively considering reinstalling the machine with ubuntu so it can run autodl-irrsi with rutorrent natively.
Problem is i am a linux noob, creating raids and smb shares are such a pain in the butt on linux. I also checked WSL2, but not sure it's performance is good enough for torrent use. But, rtorrent also runs on libtorrent.
« Last Edit: July 20, 2019, 07:58:05 PM by fusk »

Nemo

  • qBittorrent Forum
  • Administrator
  • Forum addict
  • *****
  • Posts: 1518
  • Karma: +94/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #25 on: July 20, 2019, 08:30:09 PM »
I have tried and used Biglybt too in the past for a while and I liked it. Its Vuze but without any crap build in and its being maintained. My speeds were quite good and stable too to be honest. Only downside is Java usage but most computers have minimal 8GB or more in 2019 so memory ain't a problem. But im wondering how much memory it would consume when there are 500+ torrents loaded into it for example.

I always say use whatever works the best for you ;).

Problem was that some private trackers didn't allow it, got contacted by staff that told me to switch and they wouldn't tell me why. So never knew what the issue was. But speeds were much better., but forced to switch clients.
Had a look at tixati, which also seems like a good client, but it has some weird defaults, like it stores incomplete pieces on c: and you can't adjust write cache. Ran out of ideas.

Now i am actively considering reinstalling the machine with ubuntu so it can run autodl-irrsi with rutorrent natively.
Problem is i am a linux noob, creating raids and smb shares are such a pain in the butt on linux. I also checked WSL2, but not sure it's performance is good enough for torrent use. But, rtorrent also runs on libtorrent.

The new version 2.0 says:

2019/05/27: Play nicely with private trackers and more!. Version 2.0 released.

On the private trackers I use its whitelised, before wasn't indeed. Im using qBittorrent and BiglyBT, have them both installed.
Forum Rules and Guidelines

Forum Admin.
Dutch & Turkish Translator.




fusk

  • Veteran
  • ***
  • Posts: 248
  • Karma: +4/-0
    • View Profile
Re: Torrent read/write nvme benchmark.
« Reply #26 on: July 20, 2019, 08:38:32 PM »

The new version 2.0 says:

2019/05/27: Play nicely with private trackers and more!. Version 2.0 released.

On the private trackers I use its whitelised, before wasn't indeed. Im using qBittorrent and BiglyBT, have them both installed.

I also ran 2.0, but opted in for the beta which fixed an issue with speed limits for tags. Have no idea what the problem was, it wasn't a banned client, but not whitelisted either.