Author Topic: All-time upload does not match per-torrent sum by a huge margin  (Read 423 times)

SL9

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Hello everyone,

I've been trying out qBittorrent lately, but noticed an abnormally after short usage.

Basically, the All-time upload in "View-Statistics" shows 6.08 GiB.
Whereas the sum of uploaded data of all torrents is only around 3 GiB.

See the attached screenshot for detail.
(Notice that All-time download is about 3 GiB larger than per-task sum as well.)

I can confirm that I have only ever added the four tasks as shown in the screenshot. Nothing is filtered out of view.

The only thing I can think of is protocol overhead. But I cannot imagine it being this large.

What could be the culprit? Thanks!

qBittorrent 4.1.5
Windows 7 32-bit
 
 
« Last Edit: June 22, 2019, 05:08:24 pm by SL9 »

Switeck

  • Forum addict
  • ****
  • Posts: 1467
  • Karma: +93/-0
    • View Profile
Re: All-time upload does not match per-torrent sum by a huge margin
« Reply #1 on: June 23, 2019, 09:28:07 am »
Protocol overhead really can get silly if you're on torrents with 50+ connected peers/seeds at once.
Peers/seeds regularly send HAVE messages to each other for every piece of the torrent they have, as well as PEX messages that often repeat the same known list of ip:port values of other seeds/peers.
uTP has about 5-20% packet loss on top of that, likely due to bugs in libtorrent's utp code.

Peers disconnecting and reconnecting with encrypted handshakes add a little overhead as well.

While overheads are unlikely to double the base amount uploaded to peers, it's possible for overheads to do so at worst-case scenario.
Worst-case scenario may also be due to overloads on download or upload bandwidth sides. Packet loss is guaranteed if qBT is trying to upload at unlimited speed to lots of peers at once on anything other than ridiculous-fast (1+ gbit/sec UL) fiber lines.

SL9

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: All-time upload does not match per-torrent sum by a huge margin
« Reply #2 on: June 23, 2019, 11:02:35 am »
Protocol overhead really can get silly if ..... While overheads are unlikely to double the base amount uploaded to peers, it's possible for overheads to do so at worst-case scenario.

Thank you very much for the detailed response. Now I have the big picture in mind. I'll continue using this particular copy of QBT to see if I can find some pattern. Would this delta be more related to more connected peers? uTP? Fast transfer rate? I'll test and see.

Worst-case scenario may also be due to overloads on download or upload bandwidth sides. Packet loss is guaranteed if qBT is trying to upload at unlimited speed to lots of peers at once on anything other than ridiculous-fast (1+ gbit/sec UL) fiber lines.
I am struggling to keep my ratio at some PT site so I cancelled the upload limit. But since I am "hopelessly firewalled" by my ISP I am not getting much connection for seeding. Usually one or two from time to time. For leeching, usually I am connected to 10-15 peers each with a speed of 5-10 KiB/s.

Also it's interesting to note that the "Session upload & download" stat seems close to actual torrent data transfer. So it seems that "All-time UL/DL" stat is using a different code path or logic counting the possible big overhead, whereas the session stats are not.
« Last Edit: June 23, 2019, 11:21:38 am by SL9 »

SL9

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: All-time upload does not match per-torrent sum by a huge margin
« Reply #3 on: June 23, 2019, 11:29:38 am »
Since I've started this thread it's been another 18 hours of running, the margin has widened by a little bit, now it's at:

10.46 GiB "All-time upload" vs 7.03 GiB "Actual upload". Delta: 3.24 GiB -> 3.45 GiB (0.21 GiB)
Total session upload: 3.65 GiB

90.29 GiB "All-time download" vs 86.93 GiB "Actual download". Delta: 3.16 GiB -> 3.36 GiB (0.20 GiB)
Total session download: 1.96 GiB

So the "overhead" of download and upload is pretty close.

However, the big picture of download and upload is very different. Download is capped at 32 KiB/s with ~10 peers regularly connected, while upload is mostly idle with zero peers, connected to 1 or 2 peers from time to time and creating short bursts of traffic.

Upload tranferred more data, download connected to more peers, they ended up with roughly the same amount of "overhead" I am not able to come to a conclusion the prevailing factor.

Switeck

  • Forum addict
  • ****
  • Posts: 1467
  • Karma: +93/-0
    • View Profile
Re: All-time upload does not match per-torrent sum by a huge margin
« Reply #4 on: June 24, 2019, 01:36:45 am »
"But since I am "hopelessly firewalled" by my ISP I am not getting much connection for seeding."

For that, you could get a port forwarded VPN or rent a seedbox.

"Upload tranferred more data, download connected to more peers, they ended up with roughly the same amount of "overhead" I am not able to come to a conclusion the prevailing factor."

A lot of things are far more 2-way traffic than most people realize.
Being firewalled means having to make a lot more outgoing connections (and likely encrypted handshakes) to get the same results.

fusk

  • Veteran
  • ***
  • Posts: 244
  • Karma: +4/-0
    • View Profile
Re: All-time upload does not match per-torrent sum by a huge margin
« Reply #5 on: June 30, 2019, 12:56:21 pm »
If you want to keep track of download and upload you should have a look at networx. It's awesome for that, i have a track record back to 2009.