Page 1 of 1

many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Fri Feb 14, 2020 6:06 am
by magnakai
Archlinux amd64 5.5.3-arch1-1 #1 SMP PREEMPT Tue, 11 Feb 2020 15:35:41 +0000 x86_64 GNU/Linux
4.2.1 qBittorrent
676 torrents
I turned off disk cache.
The last few days the program grow to 10G ram used.
usb external drive
The program won't shut down cleanly.
I am not sure what's wrong.
I read online turn off disk cache.
When I kill -9 it and start it back up it downs internet on the desktop until it I think checks a bunch of torrent files.
It Won't shut down cleanly.
I have moved around the async i/o threads from 1 to 128 just to experiment but put it back at 1.
I chose coalesce reads/writes.
I really am not sure why it is locking up and won't shut down cleanly.

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Fri Feb 14, 2020 4:08 pm
by magnakai
I did globally add 207 trackers and have the options in advanced for max seeding.....could that be it?

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Fri Feb 14, 2020 5:53 pm
by magnakai
It's grown to 9G ram used now on my desktop........16g ram total and its starting to make the box swap.

What can I set to make qb use less ram?
disk cache disabled
os cache off

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Sun Feb 16, 2020 1:54 am
by magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Mon Feb 17, 2020 3:58 am
by magnakai
Does anyone know why the RAM gets eaten up until box slows to snail?

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Tue Feb 18, 2020 4:01 pm
by Peter
tl;dr: your external storage is too slow. Linux is trying to help you by caching. but the storage can never catch up. resulting in huge ram usage.

- slow down your torrent client as your external HDD is clearly incapable of writing out data
- change IO scheduler for Linux
- change mount option for external drive in /etc/fstab so data gets written instantly and the write is not cached.

this is NOT a qBittorrent issue.

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Sat Feb 22, 2020 1:29 pm
by magnakai
Cmon now.

I have BFQ the best io sched for sata disk.
qb is set to not use os or disk cache.

It is most certainly a qb problem since qb is th eonyl program with dificulty.
Extermal drive writes at 70Mb /s and qb at 7

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Sat Feb 22, 2020 1:32 pm
by magnakai
root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 408 MB in 3.01 seconds = 135.72 MB/sec
[root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 512 MB in 3.00 seconds = 170.63 MB/sec
[root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 516 MB in 3.00 seconds = 171.72 MB/sec

actually external drive higher more like 150 MB /s

Why does qb eat up all the ram with cache and os cache off?
Why does it have trouble shutting down?

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Posted: Sun Feb 23, 2020 12:10 pm
by Peter
Raw sequential != random write speed lmao.

That's why HDDs always struggled with torrent clients since the dawn of time. Look up a way to measure random read/write speeds and check. It will be SUPER low because HDDs suck at it. So torrent clients download the chunks from all over the place, the OS tries to write out but HDD is slow, so the kernel/OS caches the writes because the client is set to download / uplaod too fast. Caching increases ram usage, the OS is hoping it'll be able to flush out writes at some point. But no, the user just won't let up and will not calm the client down. It keeps pushing those random writes all over the place. And in the end, all ram is consumed, cache is full, things slow to a crawl.

It's no magic or rocket science...

Ps.: Even if you disable qbt cache, the kernel/fs will try to cache file operations. Not sure if you mount your drives as I said will help. It probably will but not 100% sure.