qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Windows specific questions, problems.
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

qBT still crashes, it's just not a crash-to-desktop:
When qBT 3.0.10 reaches ~530 MB ram used, I get this weird error message:
13/07/2013 04:57:34 - Reason: Torrent file () error: Not enough space
13/07/2013 04:57:34 - An I/O error occured, 'Torrent' paused.

And no, the app won't get to much greater than 530 MB...it probably won't even reach 550 MB at 50 MB/sec. Instead high speeds seem to make the crash more severe. It can still crash-to-desktop even without opening Options when that happens.
I've had to readd the torrent a few times because qBT forgot it after crashing under those extreme tests. :(
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

Instead high speeds seem to make the crash more severe.
If you have time can you make a stacktrace(WinDbg) of this case too?
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

It's a case of I cannot duplicate on demand, only that it could rarely happen and I've seen it only 1 or 2 times in the past out of the 100's of 500+ MB memory overload attempts I've done.
I tried to trigger it again and failed. I'm sure it'll crop up again when I'm not trying.

What's important to me is that you personally have have managed to trigger the crash-to-desktop or at least the "Reason: Torrent file () error: Not enough space" via the testing method I laid out.
If you can see the problem "in action" as it were, you might have some more ideas for how to attack it, corner it, and defeat it.
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

Yes, currently I am trying to increase RAM usage to >500MB and see if it crashes.
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

I finally broke down and at least did a preliminary qBT test while running procmon. I was hoping to find a flagged event around the time of the memory overload/crash.
What I found may not be related... but I did find something rather odd:

3:09:06.5115596 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent\qBittorrent-resume.ini SUCCESS CreationTime: 7/13/2013 1:58:22 AM, LastAccessTime: 9/2/2013 3:07:23 AM, LastWriteTime: 9/2/2013 3:07:23 AM, ChangeTime: 9/2/2013 3:07:23 AM, AllocationSize: 4,096, EndOfFile: 759, FileAttributes: ANCI
3:09:06.5121887 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:06.5122778 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:06.5123706 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:06.5130581 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND
3:09:06.5131045 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:06.5131796 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND
3:09:06.5138020 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:06.5138646 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:06.5139283 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.0151895 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent\qBittorrent-resume.ini SUCCESS CreationTime: 7/13/2013 1:58:22 AM, LastAccessTime: 9/2/2013 3:07:23 AM, LastWriteTime: 9/2/2013 3:07:23 AM, ChangeTime: 9/2/2013 3:07:23 AM, AllocationSize: 4,096, EndOfFile: 759, FileAttributes: ANCI
3:09:07.0159103 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.0159983 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:07.0160935 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.0167045 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND
3:09:07.0167556 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:07.0168372 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent\qBittorrent-resume.ini PATH NOT FOUND
3:09:07.0174174 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.0174792 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:07.0175499 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\All Users\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.5156084 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent\qBittorrent-resume.ini SUCCESS CreationTime: 7/13/2013 1:58:22 AM, LastAccessTime: 9/2/2013 3:07:23 AM, LastWriteTime: 9/2/2013 3:07:23 AM, ChangeTime: 9/2/2013 3:07:23 AM, AllocationSize: 4,096, EndOfFile: 759, FileAttributes: ANCI
3:09:07.5160241 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND
3:09:07.5160940 AM qbittorrent.exe 2732 CreateFile C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: Read, Write, AllocationSize: n/a
3:09:07.5161811 AM qbittorrent.exe 2732 QueryOpen C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini NAME NOT FOUND

This is troublesome, because it may be a bug that qBT is trying to access qBittorrent.ini so often...and failing.
The actual location for qBittorrent.ini is:
C:\Documents and Settings\Switeck\Application Data\qBittorrent\qBittorrent.ini

Not
C:\Documents and Settings\Switeck\Application Data\qBittorrent.ini
or
C:\Documents and Settings\All Users\Application Data\qBittorrent.ini

Wrong file location pointers? Win XP-specific flaw? Windows-anything specific flaw?

Any chance all these failed file opens could be eating memory?
(They do seem to be occurring on 0.5 second intervals.)
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

This looks really strange. I'll run procmon too. The above segment in what time period is it? In qbt start? Close to the crash? During the crash? Or in a random moment?
PS, although setting my cache to 1000MB and cache_expiry to 600000s(internally), I never managed to make ram usage go above ~90MB (normal ~45MB) when I download an 1GB torrent... :S
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

The procmon log was from shortly before but not during a ~530 MB memory use crash. That activity seems to be happening the whole time qBT runs, which really seems excessive.

I was seeding torrents not downloading them when these crashes occurred, so it is probably the read-side cache that's filling up.
Unless you have a combination of high upload speed (like >200 KiB/sec) and/or lots of peers/upload slots (like >30) that are trying to queue up 1+ MB at a time, you probably won't trigger the crash.
But a single peer in a 100 mbit/sec LAN test should be overkill.
And a loopback test can be done on a single computer if you can run 2 BitTorrent clients at once, where qBT acts as the seeder and the other one acts as the peer.
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

I have made some progress.

1. Seeding a 2.9 torrent on qbt.
2. Setting cache to 1GB and cache expiry to 600s
3. Enabling LSD(on by default)
4. Downloading portable apps version of uttorrent
5. Firewalling everything on uttorrent except 127.0.0.1 (localhost used for LSD connections)
6. Adding same torrent on uttorrent and choosing save path.
7. Download is started and process explorer opened.

What I observed is that around 534.000KB Private bytes qbt stops the torrent and displays an error of:
03/09/2013 23:24:04 - Reason: blablabla.mkv file () error: Not enough space
Normally system errors are translated in my locale. This isn't translated so I believe that it comes from libtorrent itself. Furthermore, this error when it comes from the system is usually thrown hundred of times for the same instance and fills up the log. While this torrent has only 3 instances in the log for the same event. I highly suspect that this refers to the available memory being exhausted than the disk space.

After that the ram usage falls down at pre-seeding levels ie around 26-30MB Private bytes. I don't get a crash. I tried opening Options at around 530MB but it didn't crash.
One thing that I suspect is that libtorrent marks the cache as not swappable to pagefile/disk and maybe windows has a hard limit on that. I'll definetily check with libtorrent author in the coming days.
Your crash may be a combination of issues however. I still have to tackle the qbittorrent.ini access issue.

Note: I have 1.5GB RAM and when qbt reaches 530MB the total system usage was 1.14GB.
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

[quote="sledgehammer_999"]One thing that I suspect is that libtorrent marks the cache as not swappable to pagefile/disk and maybe windows has a hard limit on that.[/quote]Were the trigger being a hard limit on non-swappable memory use in windows, that wouldn't explain why earlier versions of qBitTorrent could get to ~1650 MB of ram use before triggering the same "Not enough space" error.
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

Please check my latest alpha build: http://qbforums.shiki.hu/index.php/topi ... ml#msg7919

Libtorrent should grow above 530MB without problem. But could you test if it obeys the limit set? eg on my machine it when I set it to 500MB it goes to 680MB. I can't test with 1GB because all I have is 1.5GB total ram. It makes my system crawl when going >1GB.
Switeck

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by Switeck »

Still crashes for me, but this time it happens around 1900-1950 MB ram used despite having the cache set to 1600 or 1700 MB.
Offset:  00252155

When I set the cache size limit to 1000 MB, qBT uses 1300 MB ram with only the single torrent loaded.
sledgehammer_999
Administrator
Administrator
Posts: 2443
Joined: Sun Jan 23, 2011 1:17 pm

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by sledgehammer_999 »

omg I have forgotten to report back to libtorrent. Although this may be a memory leak. Let's not hope it is a qbt leak.
nub

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by nub »

So any news on this guys ? it is extremely annoying, is it going to get fixed soon or i'll have to go looking for a new torrent client ?

I am getting that "Not enough space" error, cant use this client on 1gbps connection... need to have big disk cache.
loki

Re: qBT 3.0.10 >500MB cache size can trigger crash + hashfails

Post by loki »

[quote="nub"]
So any news on this guys ? it is extremely annoying, is it going to get fixed soon or i'll have to go looking for a new torrent client ?

I am getting that "Not enough space" error, cant use this client on 1gbps connection... need to have big disk cache.
[/quote]

Given that you have one post, I would assume you are a new user. Have you tried default disk cache? What if you increase the disk cache SOME, not greater than 500MB? Have you tried the latest alpha build, and what were the results? Do you have a system that can keep up with a 1gbps connection, enough RAM, fast HDD, and see different results with, for example, utorrent?

And then the general questions you should have told us... Are you on a 64bit system? What version of Windows are you using?
Post Reply