Author Topic: qBT 3.0.10 >500MB cache size can trigger crash + hashfails  (Read 20680 times)

Switeck

  • Forum addict
  • ****
  • Posts: 1518
  • Karma: +96/-0
    • View Profile
qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« on: July 13, 2013, 12:36:22 PM »
When qBT 3.0.10 reaches ~530 MB ram used, I get this weird error message:
13/07/2013 04:57:34 - Reason: Torrent file () error: Not enough space
13/07/2013 04:57:34 - An I/O error occured, 'Torrent' paused.
(Note: 'Torrent' is not actually the name of the torrent in question.)

As I tried to open the Options as qBT reached the ~530 MB ram usage limit, I got this crash:
http://www.imagebam.com/image/4431e4265215622
http://www.imagebam.com/image/beb21b265215629
http://www.imagebam.com/image/852a6d265215630

This seems to be the result of me setting qBT's write/read cache to 1000 MB size coupled with my fast upload speed of 600 KB/sec.
Neither the weird error message nor the crash occurred with the cache set to <400 MB.

The qBT 3.1.0alpha builds also exhibit this bug:
http://qbforums.shiki.hu/index.php/topic,1747.msg7138.html#msg7138

Strangely, qBT 3.0.9 experiences this bug as well, but only once it exceeds ~1600 MB ram used.
This required me setting qBT 3.0.9's cache to 1800 MB.

Remote peers may receive bad pieces as a result of this bug, as this logfile from another BT client of qBT's behavior shows:

[03:46:05]  x.x.x.x:6881 Connecting: source: C
[03:46:05]  x.x.x.x:6881 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:46:21]  x.x.x.x:6881 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed
[03:47:00]  x.x.x.x:6881 Connecting: source: C
[03:47:00]  x.x.x.x:6881 Disconnect: Connection closed
[03:47:51]  Incoming connection from x.x.x.x:4855
[03:47:51]  x.x.x.x:4855 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:47:53]  *** PIECE 787 FAILED HASH CHECK
[03:47:53]  Banned x.x.x.x:4855: [qBittorrent v3.0.10 (100.0)]: -qB30A0-q4pb%28PwLOBIx (508 MB downloaded, 3.98 MB bad)
[03:47:53]  x.x.x.x:4855 [qBittorrent v3.0.10 (100.0)]: Disconnect: Banned
[03:48:53]  Incoming connection from x.x.x.x:4858
[03:48:53]  x.x.x.x:4858 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:49:09]  x.x.x.x:4858 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed
[03:49:47]  x.x.x.x:6881 Connecting: source: I
[03:49:47]  x.x.x.x:6881 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:51:08]  x.x.x.x:6881 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed
[03:51:55]  x.x.x.x:6881 Connecting: source: I
[03:51:56]  x.x.x.x:6881 Disconnect: Peer error: No connection could be made because the target machine actively refused it.
[03:52:22]  Incoming connection from x.x.x.x:4868
[03:52:22]  x.x.x.x:4868 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:52:23]  *** PIECE 37340 FAILED HASH CHECK
[03:52:36]  x.x.x.x:4868 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed
[03:53:09]  x.x.x.x:6881 Connecting: source: I
[03:53:09]  x.x.x.x:6881 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:53:26]  x.x.x.x:6881 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed
[03:54:08]  x.x.x.x:6881 Connecting: source: I
[03:54:08]  x.x.x.x:6881 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:54:22]  x.x.x.x:6881 [qBittorrent v3.0.10 (100.0)]: Disconnect: Peer error: An existing connection was forcibly closed by the remote host.
[03:55:28]  x.x.x.x:6881 Connecting: source: I
[03:55:29]  x.x.x.x:6881 Disconnect: Peer error: No connection could be made because the target machine actively refused it.
[03:56:33]  Incoming connection from x.x.x.x:4876
[03:56:33]  x.x.x.x:4876 [qBittorrent/3.0.10.0 (0.0)]: Handshake completed
[03:56:52]  x.x.x.x:4876 [qBittorrent v3.0.10 (100.0)]: Disconnect: Connection closed

d0cent

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #1 on: July 15, 2013, 05:42:58 PM »
Same problem. Got this error on torrent i'm seeding (!)
https://dl.dropboxusercontent.com/u/10264719/qt_bug2.png
Cache size 512mb.

sledgehammer_999

  • qBittorrent maintainer
  • Administrator
  • Forum addict
  • *****
  • Posts: 2406
  • Karma: +150/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #2 on: July 15, 2013, 07:00:43 PM »
@d0cent

Can you translate the russian(?) error in your image?

Also what OS, qbittorrent version and system specs?

Dayman

  • Administrator
  • Veteran
  • *****
  • Posts: 343
  • Karma: +45/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #3 on: July 15, 2013, 08:40:02 PM »
The error is ERROR_NOACCESS (0x3E6) - Invalid access to memory location.

I've tried reproducing without success.
Building x86_64 stuff on Windows due to Gentoo syndrome.

Jabber: [email protected]
Skype: x.dayman
Irc #[email protected]: Dayman

sledgehammer_999

  • qBittorrent maintainer
  • Administrator
  • Forum addict
  • *****
  • Posts: 2406
  • Karma: +150/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #4 on: July 15, 2013, 10:17:43 PM »
As I discussed with Switeck via email, it could be a 2GB limitation on 32bit apps/os. Maybe they are hitting one of these limits, especially the ones about paged/non-paged pool.

Or in fact a genuine bug. I wonder why, our stacktrace window isn't getting triggered :S

Switeck

  • Forum addict
  • ****
  • Posts: 1518
  • Karma: +96/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #5 on: July 15, 2013, 11:18:08 PM »
The easiest way to test for this is to use qBT as a seeder to another BT client running as a peer/downloader on the same computer.
Barring running another BT client on the same computer, run another BT client on another computer on the same 100 mbit/sec network.
Wifi might work as well, but the considerably slower and unreliable speeds it gives will likely take longer to trigger the bug.
I only triggered the bug with my ~600 KiB/sec total upload speed internet connection because I had >40 peers requesting vastly different parts of a giant torrent, which the cache did considerable read-aheads -- possibly even grabbing whole 4 MB pieces when the peers were only asking for <1 MB at a time.

You'll need a torrent of >1 GB size, since qBT's cache cannot fill up unless the torrent is >500 MB in size.

It might be possible to trigger the bug with much smaller torrents if you had enough active at once, thus ruling out individual torrent size being greater than 2 GB as a cause.

During my tests leading up to the crashes...

Page Pool for qBT 3.0.10 is only about 120 K. Non-Page Pool balloons rapidly from ~9 K to ~1,300 K right before the error and/or crash.
Total Page Pool size at time of error/crash is ~1,800 K. Total Non-Page Pool is ~1600 K.
qBT's Handles, Threads, User Objects, and GDI Objects seem to remain relatively constant and low from start-up of qBT till the error and/or crash.
While even qBT 3.0.9 did not exceed the 2 GB ram usage limit, I had no expectation that it would given the settings I was using regardless of crashes.
Likewise, qBT 3.0.10 and later didn't even get close to 1 GB ram used.
My computer's free ram and free swapfile space is easily great enough through these tests that I do not have a general windows out-of-memory error, but (even if total ram free was the issue) qBT's size when the error/crash occurs would not be so predictable.
Even the exact error signature offset and exception information address in the screenshots remains remarkably consistent from crash-to-crash.
« Last Edit: July 15, 2013, 11:22:09 PM by Switeck »

d0cent

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #6 on: July 16, 2013, 03:00:30 PM »
Also what OS, qbittorrent version and system specs?
Win 7 64bit, qbt 3.0.10. 8 Gb RAM
it's 3.0.10 bug (or libtorrent) because after downgrading to 3.0.9 I see no errors.
« Last Edit: July 16, 2013, 03:05:28 PM by d0cent »

Dayman

  • Administrator
  • Veteran
  • *****
  • Posts: 343
  • Karma: +45/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #7 on: July 17, 2013, 08:13:58 PM »
I've tried reproducing this with latest RC_0_16 branch and 0.16.10 using 64-bit binary w/o any success. Perhaps it only happens with 32-bit binary.
3.0.9 - > 3.0.10 diff is not very large, so I can read it. Cache timer is manageable only in master branch, so it's not the case.

Sledge, can you try making a 3.0.10 with https://code.google.com/p/libtorrent/issues/attachmentText?id=405&aid=4050003000&name=disk_pool.diff&token=QDIiofRFzpNtXfrEFs8XJQJGOZU%3A1374087592961 reverted or 3.0.9 with latest libtorrent?
« Last Edit: July 17, 2013, 09:07:49 PM by Dayman »
Building x86_64 stuff on Windows due to Gentoo syndrome.

Jabber: [email protected]
Skype: x.dayman
Irc #[email protected]: Dayman

loki

  • Forum addict
  • ****
  • Posts: 567
  • Karma: +16/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #8 on: July 18, 2013, 03:02:24 AM »
What's the diff between 32bit builds and 64bit builds? Or differences in how they're compiled?

I'll be honest I haven't tried a 32bit build since the 64bit builds started but the 32bit builds (from sledgehammer) kept giving me some sort of I/O error if that was ever solved or not. I don't think it was RAM related at all though.

d0cent

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #9 on: July 18, 2013, 07:28:03 AM »
3.0.10 64bit version dont have this bug. Any of bugs displayed on my screenshot. No "Invalid access to memory ", no "not enough space".

Dayman

  • Administrator
  • Veteran
  • *****
  • Posts: 343
  • Karma: +45/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #10 on: July 18, 2013, 10:44:39 PM »
What's the diff between 32bit builds and 64bit builds? Or differences in how they're compiled?

There's no big difference except the architecture itself. It might be that x86_64 is still affected by the bug, but since OS limits on native 64-bit applications' memory pools are almost unreachable in daily usage (and typical desktop config), nobody is able to reproduce with 64-bit binary.

My best bet is still libtorrent changes to disk cache between 16.9 and 16.10.
Building x86_64 stuff on Windows due to Gentoo syndrome.

Jabber: [email protected]
Skype: x.dayman
Irc #[email protected]: Dayman

loki

  • Forum addict
  • ****
  • Posts: 567
  • Karma: +16/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #11 on: July 19, 2013, 12:56:03 AM »
In my experience from your latest stable 64 bit build, if anything I thought they "got it right" with disk cache in libtorrent 16.10 because I've seen less memory usage, or it actually reduces periodically instead of continuously adding. "LIBTORRENT: Cache can now be returned to the OS (0.16.10)"

But anyway, I'm off topic in this thread so just making my own observation.

BTW, disk write cache set to 128MB on mine.

sledgehammer_999

  • qBittorrent maintainer
  • Administrator
  • Forum addict
  • *****
  • Posts: 2406
  • Karma: +150/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #12 on: July 19, 2013, 01:43:04 AM »
I've tried reproducing this with latest RC_0_16 branch and 0.16.10 using 64-bit binary w/o any success. Perhaps it only happens with 32-bit binary.
3.0.9 - > 3.0.10 diff is not very large, so I can read it. Cache timer is manageable only in master branch, so it's not the case.

Sledge, can you try making a 3.0.10 with https://code.google.com/p/libtorrent/issues/attachmentText?id=405&aid=4050003000&name=disk_pool.diff&token=QDIiofRFzpNtXfrEFs8XJQJGOZU%3A1374087592961 reverted or 3.0.9 with latest libtorrent?

Yes, I provide builds.

Maybe tomorrow.

sledgehammer_999

  • qBittorrent maintainer
  • Administrator
  • Forum addict
  • *****
  • Posts: 2406
  • Karma: +150/-2
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #13 on: July 22, 2013, 02:10:04 AM »
Here are:

1. Regular build of git master with latest libtorrent RC_0_16: http://builds.shiki.hu/qbittorrent_3.1.0alpha_22072013_setup.exe
2. Git master build with latest libtorrent RC_0_16 and reverted patch linked by Dayman: http://builds.shiki.hu/temp/qbittorrent_3.1.0alpha_22072013+lb_reverted_r8171_setup.exe

Switeck

  • Forum addict
  • ****
  • Posts: 1518
  • Karma: +96/-0
    • View Profile
Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails
« Reply #14 on: July 22, 2013, 02:08:45 PM »
qbittorrent_3.1.0alpha_22072013_setup.exe
Triggered same familiar error at ~530 MB ram used:
22/07/2013 05:48:14 - Reason: Torrent file () error: Not enough space
22/07/2013 05:48:14 - An I/O error occurred, 'Torrent' paused.

qbittorrent_3.1.0alpha_22072013+lb_reverted_r8171_setup.exe
Did not get the not enough space or I/O error.
Instead it always crashed at ~1640 MB ram used.

Crash screenshots:
http://www.imagebam.com/image/b6188f266680074
http://www.imagebam.com/image/24ab55266680076
http://www.imagebam.com/image/d8a489266680079
http://www.imagebam.com/image/da0d10266680080
http://www.imagebam.com/image/758e24266680083

1st crash was nothing fancy, just let it run at whatever speed it could manage.
2nd crash was limited to 10 MB/sec speed.
3rd crash was limited to 10 MB/sec speed, and I tried to open the Options window as it reached ~1640 MB ram used. qBT crashed+disappeared, Dr Watson left running afterwords.
4th crash was without speed limits.
5th crash demonstrated the crash could happen even when limited to 1 MB/sec.

In Windows XP's Administrative Tools, Event Viewer, Application, I saw these messages:
Faulting application qbittorrent.exe, version 3.1.0.0, faulting module qbittorrent.exe, version 3.1.0.0, fault address 0x00630d87.
Faulting application qbittorrent.exe, version 3.1.0.0, faulting module qbittorrent.exe, version 3.1.0.0, fault address 0x00630d87.
Faulting application qbittorrent.exe, version 3.1.0.0, faulting module qbittorrent.exe, version 3.1.0.0, fault address 0x00163238.
Faulting application qbittorrent.exe, version 3.1.0.0, faulting module qbittorrent.exe, version 3.1.0.0, fault address 0x00630d87.
Faulting application qbittorrent.exe, version 3.1.0.0, faulting module qbittorrent.exe, version 3.1.0.0, fault address 0x00630d87.