Excessive read speed when seeding, 5-10x upload rate

Linux specific questions, problems.
Post Reply
internetaddict24

Excessive read speed when seeding, 5-10x upload rate

Post by internetaddict24 »

I thought initially that i have too many upload slots or it has a burst speed, but it's neither these, the read rate from the disk really is a few orders of magnitude higher than the upload speed.

I tried fiddling with read cache, disbaling it altogether, setting async_io to 1, nothing worked.
Downloading is perfectly fine.
Screenshot_20210406_060753.png
Screenshot_20210406_060753.png (831.45 KiB) Viewed 2112 times
OS is Arch Linux on 5.11 kernel, IO sched on drives is BFQ, they are formatted as XFS. Tried changing the io scheduler but nothing changed
unseen19
Veteran
Veteran
Posts: 188
Joined: Mon Mar 15, 2021 1:36 pm

Re: Excessive read speed when seeding, 5-10x upload rate

Post by unseen19 »

Hi,

It can comes from many sources and not only QBT. First check your libtorrent version to see if you have a current one. Their git may mention such an issue if it's related to the version or new predictive reading alg.
On the other side the HDD block size, the swap of the OS, and manu other things may influence the read speed you see.
Investigation on your memory saturation/swap usage maybe comes first in the list. Except if anyone shows up with similar symptoms you 'll have to dig a bit more your own system to reduce the scope
internetaddict24

Re: Excessive read speed when seeding, 5-10x upload rate

Post by internetaddict24 »

Yeah indeed now i found out about the predictive read algorythm that was supposed to work together with suggest_mode, but ends up reading excessively even when the disk cache is full and it's got decent amount of hits (15-20%)

I will raise an issue at libtorrent's git and return here with what i found out

After making a clean reinstall of libtorrent (1.2.11), erasing settings files, setting cache to 0 and suggest_mode to 0 now the reads are fine. Not sure yet about asyc_io but i think that has nothing to do here, i set it at 1 just in case for now.

I have a curiosity though: With the OS cache enabled, does it also cache reads, or only the writes? Or does it depend on the OS? Is libtorrent's cache simply a separate cache like a fallback in case the OS doesn't cache anything?
unseen19
Veteran
Veteran
Posts: 188
Joined: Mon Mar 15, 2021 1:36 pm

Re: Excessive read speed when seeding, 5-10x upload rate

Post by unseen19 »

Interesting question. Here is what I think right (best to confirm): the OS cache is caching files, Libtorrent torrent pieces only.
Of course The second may benefit from the first, but it depends. I remember some years ago when it was advisable to disable the OS cache on Windows if you use it as an always on seedbox. The Linux caching was not involved if I'm not wrong

BTW I used to configure libtorrent but when testing QBT I saw the default value for the cache was "-1", so some kind of auto mode. How does it work with a Linux seedbox for example? My goal is Ubuntu on RPI4 and as you tested the settings...My setup was 4.3.3/1.2.12 and with all default it ran well even with the desktop UI and realvnc to access the full GUI (I'm not happy with only the webui, some missing options for my usage)
internetaddict24

Re: Excessive read speed when seeding, 5-10x upload rate

Post by internetaddict24 »

Update:
Libtorrent 1.12.14 appears to ameliorate this issue. I'm still yet to play around with suggest_mode, but all the other options i previously had enabled are now working fine again.

They are piece_extent_affinity=1, 105% watermark ratio, 2048k watermark, 2 GB disk cache. These are fine so far.
unseen19 wrote: Wed Apr 07, 2021 3:58 pm Interesting question. Here is what I think right (best to confirm): the OS cache is caching files, Libtorrent torrent pieces only.
Of course The second may benefit from the first, but it depends. I remember some years ago when it was advisable to disable the OS cache on Windows if you use it as an always on seedbox. The Linux caching was not involved if I'm not wrong

BTW I used to configure libtorrent but when testing QBT I saw the default value for the cache was "-1", so some kind of auto mode. How does it work with a Linux seedbox for example? My goal is Ubuntu on RPI4 and as you tested the settings...My setup was 4.3.3/1.2.12 and with all default it ran well even with the desktop UI and realvnc to access the full GUI (I'm not happy with only the webui, some missing options for my usage)
Replying two months later :)
From what i found out the libtorrent cache is also being used for any random reads which aids in seeding. The OS cache is in Linux mostly used for writes. So it is wise to use a combination between the two, i would say at least 256 mb + OS cache enabled. I'm using 1-2 gb because i'm seeding many torrents with many active peers, without it random reads will torture the drives.
Setting cache to -1 i'm not sure if it calculates to an eigth of the ram or not, i think it's dynamic now.

I also had a seedbox on a low power pc and exactly VNC i used to remote into it, it's a good method. Just be sure to use lxde to avoid high gui overhead/mem usage
unseen19
Veteran
Veteran
Posts: 188
Joined: Mon Mar 15, 2021 1:36 pm

Re: Excessive read speed when seeding, 5-10x upload rate

Post by unseen19 »

replying two months later :)
From what i found out the libtorrent cache is also being used for any random reads which aids in seeding. The OS cache is in Linux mostly used for writes. So it is wise to use a combination between the two, i would say at least 256 mb + OS cache enabled. I'm using 1-2 gb because i'm seeding many torrents with many active peers, without it random reads will torture the drives.
Setting cache to -1 i'm not sure if it calculates to an eigth of the ram or not, i think it's dynamic now.

I also had a seedbox on a low power pc and exactly VNC i used to remote into it, it's a good method. Just be sure to use lxde to avoid high gui overhead/mem usage
I paused my project but will soon restart it to these info are welcome. I'll test this way, including VNC recommandation.
Thank you for the feedback :)
Post Reply