Using cache actually increases disk wear/reads?

Windows specific questions, problems.
Post Reply
user637745

Using cache actually increases disk wear/reads?

Post by user637745 »

Im not sure if its ok to post a question in both "generic" and "windows" section, if not i apologize and you can remove this.

So i have been monitoring disk reads with iotop and psexec in windows, and with cache enabled, the amount of reads from disk is quite alot higher with cache enabled in qbittorrent.

Can anyone shed some light why this is?
User avatar
Peter
Administrator
Administrator
Posts: 2693
Joined: Wed Jul 07, 2010 6:14 pm

Re: Using cache actually increases disk wear/reads?

Post by Peter »

Phew this is a very in-depth question and I am fairly sure, only the creator of libtorrent could shed some light on it.
user637745

Re: Using cache actually increases disk wear/reads?

Post by user637745 »

Peter wrote: Tue Nov 10, 2020 3:39 pm Phew this is a very in-depth question and I am fairly sure, only the creator of libtorrent could shed some light on it.
Ok thank you, can anyone else confirm what im seeing? Im trying to find proof that its not like im saying :P
SideshowBob

Re: Using cache actually increases disk wear/reads?

Post by SideshowBob »

I don't know how much difference there is between the Windows and the Unix versions, but I have options for cache (off by default) and OS caching (on by default). I presume it's not a good idea to have both because it could result in files being cached twice, leading to less effective caching and more disk access.

If you have cache turned-on, but turn-off OS caching, that may lead to less memory being available for caching. One of the reasons for switching to application level caching could be to reduce impact of OS cache use on other processes.

If this a general purpose computer doing other things, I suspect you would be better off leaving it to the OS.
user637745

Re: Using cache actually increases disk wear/reads?

Post by user637745 »

SideshowBob wrote: Fri Nov 13, 2020 2:05 am I don't know how much difference there is between the Windows and the Unix versions, but I have options for cache (off by default) and OS caching (on by default). I presume it's not a good idea to have both because it could result in files being cached twice, leading to less effective caching and more disk access.

If you have cache turned-on, but turn-off OS caching, that may lead to less memory being available for caching. One of the reasons for switching to application level caching could be to reduce impact of OS cache use on other processes.

If this a general purpose computer doing other things, I suspect you would be better off leaving it to the OS.
I have been having OS cache off in all tests, still alot more disk reads
SideshowBob

Re: Using cache actually increases disk wear/reads?

Post by SideshowBob »

user637745 wrote: Fri Nov 13, 2020 7:36 am I have been having OS cache off in all tests, still a lot more disk reads
Right, but that probably reduces the amount of memory available for caching compared with an OS cache. The OS cache can use almost all the memory this isn't doing something more important. If you give a similar amount to an application level cache you run the risk of swap usage.

I should emphasize that I have no particular knowledge of qBittorrent's caching. I'm speculating from general principles.
slybunda

Re: Using cache actually increases disk wear/reads?

Post by slybunda »

qbittorrent cache doesnt work that well since its based on libtorrent. it will disk thrash on mechanical hdd. utorrents cache system works far better on mechanical drives since it buffers writes into larger blocks to write them out as a sequential write. you will hear the difference with utorrent when downloading same torrent vs qbitorrent. far less disk thrashing with utorrent.
User avatar
Peter
Administrator
Administrator
Posts: 2693
Joined: Wed Jul 07, 2010 6:14 pm

Re: Using cache actually increases disk wear/reads?

Post by Peter »

slybunda wrote: Fri Dec 25, 2020 10:48 am qbittorrent cache doesnt work that well since its based on libtorrent. it will disk thrash on mechanical hdd. utorrents cache system works far better on mechanical drives since it buffers writes into larger blocks to write them out as a sequential write. you will hear the difference with utorrent when downloading same torrent vs qbitorrent. far less disk thrashing with utorrent.
for me it worked/works the same as uT. both increased HDD write speeds for me significantly. though I've only used cache sizes up to ~1gb.
slybunda

Re: Using cache actually increases disk wear/reads?

Post by slybunda »

Its to do with block size where qbittorrent uses 16k and uTorrent uses 132k or so. On ssd qbittorrent can end up high amount of write amplification
Post Reply