Page 1 of 1
Using cache actually increases disk wear/reads?
Posted: Tue Nov 10, 2020 2:56 pm
by user637745
Im not sure if its ok to post a question in both "generic" and "windows" section, if not i apologize and you can remove this.
So i have been monitoring disk reads with iotop and psexec in windows, and with cache enabled, the amount of reads from disk is quite alot higher with cache enabled in qbittorrent.
Can anyone shed some light why this is?
Re: Using cache actually increases disk wear/reads?
Posted: Tue Nov 10, 2020 3:39 pm
by Peter
Phew this is a very in-depth question and I am fairly sure, only the creator of libtorrent could shed some light on it.
Re: Using cache actually increases disk wear/reads?
Posted: Thu Nov 12, 2020 12:22 pm
by user637745
Peter wrote: ↑Tue Nov 10, 2020 3:39 pm
Phew this is a very in-depth question and I am fairly sure, only the creator of libtorrent could shed some light on it.
Ok thank you, can anyone else confirm what im seeing? Im trying to find proof that its not like im saying
Re: Using cache actually increases disk wear/reads?
Posted: Fri Nov 13, 2020 2:05 am
by SideshowBob
I don't know how much difference there is between the Windows and the Unix versions, but I have options for cache (off by default) and OS caching (on by default). I presume it's not a good idea to have both because it could result in files being cached twice, leading to less effective caching and more disk access.
If you have cache turned-on, but turn-off OS caching, that may lead to less memory being available for caching. One of the reasons for switching to application level caching could be to reduce impact of OS cache use on other processes.
If this a general purpose computer doing other things, I suspect you would be better off leaving it to the OS.
Re: Using cache actually increases disk wear/reads?
Posted: Fri Nov 13, 2020 7:36 am
by user637745
SideshowBob wrote: ↑Fri Nov 13, 2020 2:05 am
I don't know how much difference there is between the Windows and the Unix versions, but I have options for cache (off by default) and OS caching (on by default). I presume it's not a good idea to have both because it could result in files being cached twice, leading to less effective caching and more disk access.
If you have cache turned-on, but turn-off OS caching, that may lead to less memory being available for caching. One of the reasons for switching to application level caching could be to reduce impact of OS cache use on other processes.
If this a general purpose computer doing other things, I suspect you would be better off leaving it to the OS.
I have been having OS cache off in all tests, still alot more disk reads
Re: Using cache actually increases disk wear/reads?
Posted: Fri Nov 13, 2020 10:51 pm
by SideshowBob
user637745 wrote: ↑Fri Nov 13, 2020 7:36 am
I have been having OS cache off in all tests, still a lot more disk reads
Right, but that probably reduces the amount of memory available for caching compared with an OS cache. The OS cache can use almost all the memory this isn't doing something more important. If you give a similar amount to an application level cache you run the risk of swap usage.
I should emphasize that I have no particular knowledge of qBittorrent's caching. I'm speculating from general principles.
Re: Using cache actually increases disk wear/reads?
Posted: Fri Dec 25, 2020 10:48 am
by slybunda
qbittorrent cache doesnt work that well since its based on libtorrent. it will disk thrash on mechanical hdd. utorrents cache system works far better on mechanical drives since it buffers writes into larger blocks to write them out as a sequential write. you will hear the difference with utorrent when downloading same torrent vs qbitorrent. far less disk thrashing with utorrent.
Re: Using cache actually increases disk wear/reads?
Posted: Fri Dec 25, 2020 11:28 am
by Peter
slybunda wrote: ↑Fri Dec 25, 2020 10:48 am
qbittorrent cache doesnt work that well since its based on libtorrent. it will disk thrash on mechanical hdd. utorrents cache system works far better on mechanical drives since it buffers writes into larger blocks to write them out as a sequential write. you will hear the difference with utorrent when downloading same torrent vs qbitorrent. far less disk thrashing with utorrent.
for me it worked/works the same as uT. both increased HDD write speeds for me significantly. though I've only used cache sizes up to ~1gb.
Re: Using cache actually increases disk wear/reads?
Posted: Fri Dec 25, 2020 5:41 pm
by slybunda
Its to do with block size where qbittorrent uses 16k and uTorrent uses 132k or so. On ssd qbittorrent can end up high amount of write amplification