Author Topic: qbt with lt 1.1 has worse torrent rechecking/creating speed on Windows  (Read 472 times)

chaoticbeliever

  • Member
  • **
  • Posts: 65
  • Karma: +0/-0
    • View Profile
This issue identifies a probably libtorrent-side performance degradation wrt torrent rechecking on Windows. Maybe this one should be forwarded to libtorrent, but I think it is better be forwarded by qbt main devs or I will do it later.

qBittorrent version and Operating System
Windows 10 1709/1803, but I think it is general to at least Windows 7+
qBittorrent 3.3.16, 4.0.3, 4.0.4, 4.1.0, 4.1.1, but I think it's general to qbt 3.3+ and 4+
libtorrent: 1.0.11 (at 62c9679) and 1.1.7 but I think it's general to 1.0.6+ and 1.1+

What is the problem
I made a benchmark on torrent rechecking/creating wrt qbt 4.1.1 + lt 1.0.11/1.1.7, additionally with uT2.2.1 for comparison. From the result below, qBittorrent 4.1.1 will have a worse torrent rechecking/creating performance using libtorrent 1.1.7. This slower speed is actually general to any qBittorrent 3.3+ and 4+ built against libtorrent 1.1.x. If built against libtorrent 1.0.x, e.g. 1.0.11, qBittorrent will have a much better speed on that. The potential disk read and hash capability of libtorrent 1.0.11 should be much higher than uTorrent, as seen by 1000MB/s creating speed on NVMe SSD, but there is a degradation on rechecking, and a further drive-dependent degradation as seen on the SATA3 SSD, not seen by uTorrent.

torrent creating
speed / active time   NVMe SSD
SM951   SATA SSD
X400   3.5" HDD
ST8000DM   2.5" HDD
ST2000LM
4.1.1+1.1.7   260-280MB/s
100%   130-150MB/s
~60%   80-85MB/s*
~80%   130-135MB/s
~60%
4.1.1+1.0.11   ~1.0GB/s
100%   460-480MB/s
~85%   195-205MB/s
~97%   130-135%
~96%
2.2.1   450-470MB/s
100%   450-470MB/s
~90%   195-200MB/s
90-95%   130-135MB/s
~95%
torrent rechecking
speed / active time   NVMe SSD
SM951   SATA SSD
X400   3.5" HDD
ST8000DM   2.5" HDD
ST2000LM
4.1.1 + 1.1.7   260-280MB/s
100%   130-150MB/s
~60%   80-85MB/s*
~80%   130-135MB/s
~60%
4.1.1 + 1.0.11   550-620MB/s
~30%   230-280MB/s
~65%   195-205MB/s
~76%   130-135MB/s
~85%
2.2.1   465-475MB/s
100%   440-460MB/s
~90%   195-200MB/s
90-95%   130-135MB/s
~95%
*The performance here is strange, but I tested it multiple times and the value is true. I have no idea why qbt4.1.1+lt1.1.7 should be slower on ST8000 than ST2000.

4.1.1+1.1.7 = the qbt official x64 build of qbt 4.1.1, lt 1.1.7, boost 1.67.0, Qt 5.10.1
4.1.1+1.0.11 = my x64 build of qbt 4.1.1, lt 1.0.11 (at 62c9679), boost 1.65.1, Qt 5.10.1
2.2.1 = uTorrent 2.2.1 build 25302
all using default configuration

Sequential read capability (the outmost cylinder if HDD):
SM951 = Samsung SM951 512G MLC M.2 NVMe SSD, ~1500MB/s, internal PCIe 3.0 4X
X400 = SanDisk X400 1TB TLC M.2 SATA3 SSD, ~500MB/s, internal SATA3
ST8000 = Seagate ST8000DM004, 3.5" 8T 5425R 256M, ~200MB/s, mounted via USB3.0
ST2000 = Seagate ST2000LM003, 2.5" 2T 5400R 32M, ~135MB/s, internal SATA3
CPU is 6700K, RAM 48GB

Test is based on 30GB large files (typ. 2GB each file), in 4MiB piece size
Speed and active time is the value shown in Task Manager
Both HDDs are reformated before test

What is the expected behaviour
Libtorrent 1.1.x should at least have a same disk perf on creating/rechecking torrent as 1.0.x.
Maybe qBittorrent should re-consider the libtorrent version to be used before libtorrent 1.1 becomes better.

Steps to reproduce
You need at least an SSD (assume you are not using some "flash disk" level SSD of 300MB/s or lower). For comparison, lt 1.1x group can include any official qbt 4.x builds (which are all built again lt 1.1.x), and lt 1.0.x group can include the official qbt 3.3.16 build (which is on lt 1.0.11) and my builds of qbt 4.1.1 against lt 1.0.11. You can also build other qbt+lt combination. My note here is that lt 1.0.11 at 4e90eb1 (the one in the releases page) or lower seems not to compile with boost 1.65.0 or above, but 1.64.0 is fine; lt 1.0.11 at 62c9679 does not compile with boost 1.66.0 or above, but 1.65.1 is fine.

It should also be noted that due to windows system read cache, before creating/rechecking you should do enough other read/write jobs to wipe the torrent content off the memory, otherwise operation will directly hit cache from memory at a high speed.

Extra info(if any)
I can upload all screenshots later (maybe in one or two days) if necessary.

As for Linux, I did not perform any similar test as I always compile libtorrent 1.0.11 on it, but if necessary maybe I can test it on my seedboxes, but this might need 1 week or more.

Previously I noticed the discussion about Windows IO and cache issue, so I also conducted some investigation on related setting about libtorrent session (typically becomes settings_pack in 1.1 branch), I altered settings including enable_os_cache, low_priority_disk, coalesce_reads (by qBittorrent GUI or modifying qbt source code) but result does not show any change. This needs further investigation.

I have been trying to fix this issue, but I don't think I can do it faster or better than current main qbt and lt devs. If main devs are willing to fix this issue, I can help run benchmarks.