many torrent on external drive: RAM usage killing my Archlinux desktop help!

Linux specific questions, problems.
Post Reply
magnakai

many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

Archlinux amd64 5.5.3-arch1-1 #1 SMP PREEMPT Tue, 11 Feb 2020 15:35:41 +0000 x86_64 GNU/Linux
4.2.1 qBittorrent
676 torrents
I turned off disk cache.
The last few days the program grow to 10G ram used.
usb external drive
The program won't shut down cleanly.
I am not sure what's wrong.
I read online turn off disk cache.
When I kill -9 it and start it back up it downs internet on the desktop until it I think checks a bunch of torrent files.
It Won't shut down cleanly.
I have moved around the async i/o threads from 1 to 128 just to experiment but put it back at 1.
I chose coalesce reads/writes.
I really am not sure why it is locking up and won't shut down cleanly.
magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

I did globally add 207 trackers and have the options in advanced for max seeding.....could that be it?
magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

It's grown to 9G ram used now on my desktop........16g ram total and its starting to make the box swap.

What can I set to make qb use less ram?
disk cache disabled
os cache off
magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

Does anyone know why the RAM gets eaten up until box slows to snail?
User avatar
Peter
Administrator
Administrator
Posts: 2693
Joined: Wed Jul 07, 2010 6:14 pm

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by Peter »

tl;dr: your external storage is too slow. Linux is trying to help you by caching. but the storage can never catch up. resulting in huge ram usage.

- slow down your torrent client as your external HDD is clearly incapable of writing out data
- change IO scheduler for Linux
- change mount option for external drive in /etc/fstab so data gets written instantly and the write is not cached.

this is NOT a qBittorrent issue.
magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

Cmon now.

I have BFQ the best io sched for sata disk.
qb is set to not use os or disk cache.

It is most certainly a qb problem since qb is th eonyl program with dificulty.
Extermal drive writes at 70Mb /s and qb at 7
magnakai

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by magnakai »

root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 408 MB in 3.01 seconds = 135.72 MB/sec
[root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 512 MB in 3.00 seconds = 170.63 MB/sec
[root@tax-rent-only ~]# hdparm -t /dev/sdc1

/dev/sdc1:
Timing buffered disk reads: 516 MB in 3.00 seconds = 171.72 MB/sec

actually external drive higher more like 150 MB /s

Why does qb eat up all the ram with cache and os cache off?
Why does it have trouble shutting down?
User avatar
Peter
Administrator
Administrator
Posts: 2693
Joined: Wed Jul 07, 2010 6:14 pm

Re: many torrent on external drive: RAM usage killing my Archlinux desktop help!

Post by Peter »

Raw sequential != random write speed lmao.

That's why HDDs always struggled with torrent clients since the dawn of time. Look up a way to measure random read/write speeds and check. It will be SUPER low because HDDs suck at it. So torrent clients download the chunks from all over the place, the OS tries to write out but HDD is slow, so the kernel/OS caches the writes because the client is set to download / uplaod too fast. Caching increases ram usage, the OS is hoping it'll be able to flush out writes at some point. But no, the user just won't let up and will not calm the client down. It keeps pushing those random writes all over the place. And in the end, all ram is consumed, cache is full, things slow to a crawl.

It's no magic or rocket science...

Ps.: Even if you disable qbt cache, the kernel/fs will try to cache file operations. Not sure if you mount your drives as I said will help. It probably will but not 100% sure.
Post Reply