All DHT nodes are dropped after 30m+ uptime

[Dayman]

I have a strange situation. When i start qbt i see 50+ DHT nodes, but after qbt runs for more than 30 minutes all DHT nodes are dropped (status bar says "DHT: 0").
DHT/PeX/uTP are all enabled. Both udp and tcp ports are forwarded through NAT.
Can anybody confirm this behavior?

[ironcross]

[quote="Dayman"]
Can anybody confirm this behavior?
[/quote]

Negative.
UPnP/NAT-PMP enabled.
uTP enabled.
DHT, PEX, Local peer discovery enabled.
DHT on the same port as BitTorrent.
More than 30 mins. running.
DHT: 176 nodes.

[pragmaticenigma]

I have been running qBittorrent for several days and have not seen DHT nodes dropped.

Are you using alternate port numbers for your client? Make sure you are not using the default port numbers or any port number in the 6000's as these are known ports that Internet Service Providers (ISP) commonly block.

Also, make sure if you are using any firewalls that you have the ports open to allow connections.

Finally, make sure your router is setup to forward the ports from your Internet connection to the computer running qBittorrent.

You might also want to make sure DHT is not using the same port as BitTorrent.  This setting is found in Preferences under Bitorroent. Check the box for "Usa a different port. . ." and set the port a few numbers higher or lower than your BitTorrent port.  Make sure you allow these ports through any firewalls!

[Dayman]

https://bugs.launchpad.net/qbittorrent/+bug/999390 and here http://qbforums.shiki.hu/index.php?topic=324.0
Looks like I'm not the only one.

[Dayman]

Looks like i've managed to narrow down the problem (lol, this phrase doesn't make any sense atm).
What i've tried so far:

1. Separate port for DHT -> same result
2. Remove router from the chain (connect directly) -> same result
3. Connect through router but from Linux (Gentoo Hardened x86 qBt 2.9.9 libtorrent 0.16.1 Boost 1.49 Qt 4.8.1) -> WORKS
4. Use Virtual Machine with XP Pro x64 and custom built qBt (VM -> bridge -> Win7 -> router) -> WORKS

So yeah, the router works fine and is not the culprit. The problem is either in my

Code: Select all

-DEFINES += _WIN32_WINNT=0x0500
+DEFINES += _WIN32_WINNT=0x0601

which probably is not the case, since official 2.9.7 build has same problem for me
OR
somewhere in Win7 <- that's where i've managed to narrow this =)

[pragmaticenigma]

I have had strange issues as well with Win7 and networking.  What mode is your network setup for on Windows 7 (Public, Work, Home)?

I had issues where if setup the network for Public that any applications trying to use open UDP ports would become blocked after a certain amount of time.  The only solution I found was making sure that my network connection is setup as Home and the ports opened, then I no longer had any issue.

Are you using any VPN connections for your BitTorrenting?

[loki]

Yeah, mine consistently says DHT: 1.
I too have Win7, my network type is set as home, and connected by ethernet from router. I'm leaning towards something related to Win7...

[pragmaticenigma]

Reading another forum, it appears that removing the firewall rules for the BitTorrent ports and re-adding them has helped some people.

[ironcross]

[quote="pragmaticenigma"]
What mode is your network setup for on Windows 7 (Public, Work, Home)?
Are you using any VPN connections for your BitTorrenting?
[/quote]

Home. But it's really a home network with a router, 2 PCs and 1 laptop.
No VPN connections.

[Dayman]

I'm at a loss. I moved DHT to another port, so i don't get uTP traffic in tcpdump.
And there's goddamn nothing.
Nevertheless I was able to receive incoming udp with PFportchecker and log it with tcpdump. That's damn strange.

EDIT:
Still no results. Strange thing is every other client, which gives DHT info works fine and qBt works fine inside VM with bridged networking (wtf? really?). I've already tried to disable every kind of checksum offloading with no result again.
In wireshark for qBt i mostly see

Code: Select all

uTorrent Transport Protocol Type: Unknown [0-9]{1,3}

with uTorrent i mostly see

Code: Select all

uTorrent Transport Protocol Fin

with a very few

Code: Select all

uTorrent Transport Protocol Type: Unknown [0-9]{1,3}

[pragmaticenigma]

I'm just confused as to what might be causing this...

Only other thought I have is anti-virus or anti-malware program is making adjustments to the window firewall and causing dropped packets...

I wouldn't think much of the wireshark results, almost all Bittorrent clients rely on the same core torrent library so it would be really strange for one client to work while another fails.  Personally I like uTorrent, but I prefer to run linux and uTorrent does not have a linux GUI client (I find web guis cumbersome)

[Dayman]

As a last resort i will try building qBt with mingw (x86_64 experimental one https://code.google.com/p/mingw-builds/). OpenSSL builds in MSYS and passes all tests and that's a huge step forward already (I remember building OpenSSL for WinCE: that was something).

P.S. Had to drop '-flto'. Seems to be broken on Windows.

EDIT
Ok. Everything builds and runs with mingw64 (not without crutches, but still). And nevertheless DHT still doesn't work (thus, i'm declaring win7 a piece of [****])
If anyone is interested in building on mingw64, there are some bugs.
First of all libtorrent won't build as static library: looks like cmd line args overflow - symbols just get swallowed in the middle; no way around this it seems, has to be build as dll.
Second libtorrent needs to be patched to even build.

Code: Select all

diff -ru libtorrent_old/include/libtorrent/config.hpp libtorrent_new/include/libtorrent/config.hpp
--- libtorrent_old/include/libtorrent/config.hpp	2012-04-28 20:23:43.000000000 +0400
+++ libtorrent_new/include/libtorrent/config.hpp	2012-06-30 00:48:19.670249700 +0400
@@ -473,7 +473,7 @@
 
 #if defined __APPLE__ && defined __MACH__
 #define TORRENT_USE_ABSOLUTE_TIME 1
-#elif defined(_WIN32) || defined TORRENT_MINGW
+#elif defined(_WIN32) || defined (_WIN64) || defined TORRENT_MINGW
 #define TORRENT_USE_QUERY_PERFORMANCE_TIMER 1
 #elif defined(_POSIX_MONOTONIC_CLOCK) && _POSIX_MONOTONIC_CLOCK >= 0
 #define TORRENT_USE_CLOCK_GETTIME 1
diff -ru libtorrent_old/include/libtorrent/stat.hpp libtorrent_new/include/libtorrent/stat.hpp
--- libtorrent_old/include/libtorrent/stat.hpp	2012-03-27 06:03:11.000000000 +0400
+++ libtorrent_new/include/libtorrent/stat.hpp	2012-06-30 00:52:09.144374900 +0400
@@ -42,6 +42,9 @@
 #include "libtorrent/invariant_check.hpp"
 #include "libtorrent/config.hpp"
 #include "libtorrent/assert.hpp"
+#ifdef __MINGW64__
+#include <sys/stat.h>
+#endif
 
 #if defined TORRENT_VERBOSE_LOGGING || defined TORRENT_LOGGING || defined TORRENT_ERROR_LOGGING
 #include "libtorrent/debug.hpp" // for logger
diff -ru libtorrent_old/src/asio_ssl.cpp libtorrent_new/src/asio_ssl.cpp
--- libtorrent_old/src/asio_ssl.cpp	2012-04-12 04:37:09.000000000 +0400
+++ libtorrent_new/src/asio_ssl.cpp	2012-06-30 00:51:01.500505900 +0400
@@ -2,6 +2,9 @@
 #include <boost/version.hpp>
 
 #if BOOST_VERSION >= 104610
+#ifdef __MINGW64__
+#include <limits.h>
+#endif
 #include <boost/asio/ssl/impl/src.hpp>
 #endif
 
diff -ru libtorrent_old/src/enum_net.cpp libtorrent_new/src/enum_net.cpp
--- libtorrent_old/src/enum_net.cpp	2012-05-13 22:54:34.000000000 +0400
+++ libtorrent_new/src/enum_net.cpp	2012-06-30 00:50:06.948385700 +0400
@@ -63,6 +63,9 @@
 #define WIN32_LEAN_AND_MEAN
 #endif
 #include <windows.h>
+#ifdef __MINGW64__
+#include <wincrypt.h>
+#endif
 #include <iphlpapi.h>
 #endif
 
@@ -886,7 +889,7 @@
 			return std::vector<ip_route>();
 		}
 
-#if _WIN32_WINNT >= 0x0600
+#if _WIN32_WINNT >= 0x0600 && ! defined (__MINGW32__)
 		typedef DWORD (WINAPI *GetIpForwardTable2_t)(
 			ADDRESS_FAMILY, PMIB_IPFORWARD_TABLE2*);
 		typedef void (WINAPI *FreeMibTable_t)(PVOID Memory);
diff -ru libtorrent_old/src/file_pool.cpp libtorrent_new/src/file_pool.cpp
--- libtorrent_old/src/file_pool.cpp	2012-04-25 08:37:28.000000000 +0400
+++ libtorrent_new/src/file_pool.cpp	2012-06-30 00:50:06.807377600 +0400
@@ -155,7 +155,7 @@
 				}
 #ifdef TORRENT_WINDOWS
 // file prio is supported on vista and up
-#if _WIN32_WINNT >= 0x0600
+#if _WIN32_WINNT >= 0x0600 && ! defined (__MINGW32__)
 				if (m_low_prio_io)
 				{
 					// TODO: load this function dynamically from Kernel32.dll
diff -ru libtorrent_old/src/time.cpp libtorrent_new/src/time.cpp
--- libtorrent_old/src/time.cpp	2012-03-27 06:02:55.000000000 +0400
+++ libtorrent_new/src/time.cpp	2012-06-30 00:48:19.780256000 +0400
@@ -38,7 +38,7 @@
 #include "libtorrent/config.hpp"
 #include "libtorrent/time.hpp"
 
-#ifndef _WIN32
+#ifndef _WIN32 || not defined _WIN64
 #include <unistd.h>
 #endif
 

And (to build against --layout=system boost)

Code: Select all

diff -ru libtorrent/Jamfile libtorrent2/Jamfile
--- libtorrent/Jamfile	2012-06-10 20:15:02.000000000 +0400
+++ libtorrent2/Jamfile	2012-06-29 17:42:20.122327200 +0400
@@ -179,6 +179,9 @@
 	else
 	{
 		result += <library>boost_system ;
+		result += <library>boost_date_time ;
+		result += <library>boost_filesystem ;
+		result += <library>boost_thread ;
 
 		# on mac the boost headers are installed in
 		# a directory that isn't automatically accessable
@@ -387,8 +390,8 @@
 feature export-extra : off on : composite propagated ;
 
 # required for openssl on windows
-lib ssleay32 : : <name>ssleay32 ;
-lib libeay32 : : <name>libeay32 ;
+lib ssleay32 : : <name>crypto.dll ;
+lib libeay32 : : <name>ssl.dll ;
 lib advapi32 : : <name>Advapi32 ;
 lib user32 : : <name>User32 ;
 lib shell32 : : <name>shell32 ;
@@ -415,6 +418,9 @@
 lib boost_system : : <target-os>solaris <name>boost_system $(boost-library-search-path) ;
 
 lib boost_system : : <name>boost_system ;
+lib boost_date_time : : <name>boost_date_time ;
+lib boost_filesystem : : <name>boost_filesystem ;
+lib boost_thread : : <name>boost_thread ;
 
 # openssl on linux/bsd/macos etc.
 lib gcrypt : : <name>gcrypt <link>shared <search>/opt/local/lib ;
@@ -543,6 +549,7 @@
 	<variant>debug:<define>TORRENT_DEBUG
 	<define>_FILE_OFFSET_BITS=64
 	<define>BOOST_EXCEPTION_DISABLE
+    <define>BOOST_ALL_NO_LIB
 # enable cancel support in asio
 	<define>BOOST_ASIO_ENABLE_CANCELIO
 	<conditional>@linking

Third. Libtorrent must be built with export-extra=on and linked against with -DTORRENT_EXPORT_EXTRA, otherwise you will get unresolved externals when linking qBt.
Lastly qBt needs to be built with the following config:

Code: Select all

diff -ru q1/winconf-mingw.pri q2/winconf-mingw.pri
--- q1/winconf-mingw.pri	2012-06-24 17:50:22.000000000 +0400
+++ q2/winconf-mingw.pri	2012-06-30 01:46:15.490178900 +0400
@@ -6,17 +6,19 @@
 
 #Adapt the lib names/versions accordingly
 CONFIG(debug, debug|release) {
-  LIBS += libtorrent \
-          libboost_system-mgw45-mt-d-1_47 \
-          libboost_filesystem-mgw45-mt-d-1_47 \
-          libboost_thread-mgw45-mt-d-1_47
+  LIBS += libtorrent.dll \
+          libboost_system \
+          libboost_filesystem \
+          libboost_thread \
+          libboost_date_time
 } else {
-  LIBS += libtorrent \
-          libboost_system-mgw45-mt-1_47 \
-          libboost_filesystem-mgw45-mt-1_47 \
-          libboost_thread-mgw45-mt-1_47
+  LIBS += libtorrent.dll \
+          libboost_system \
+          libboost_filesystem \
+          libboost_thread \
+          libboost_date_time
 }
 
 LIBS += libadvapi32 libshell32
-LIBS += libcrypto.dll libssl.dll libwsock32 libws2_32 libz libiconv.dll
+LIBS += libcrypto.dll libssl.dll libwsock32 libws2_32 libzlibstatic
 LIBS += libpowrprof
diff -ru q1/winconf.pri q2/winconf.pri
--- q1/winconf.pri	2012-06-24 17:50:22.000000000 +0400
+++ q2/winconf.pri	2012-06-30 10:34:59.331092200 +0400
@@ -1,29 +1,32 @@
 # Adapt these paths on Windows
 
 #Point this to the boost include folder
-INCLUDEPATH += $$quote(C:/qBittorrent/boost_1_46_0)
+INCLUDEPATH += $$quote(T:/_outdir/Boost/Boost64_gcc/include)
 #Point this to the libtorrent include folser
-INCLUDEPATH += $$quote(C:/qBittorrent/RC_0_15/include)
+INCLUDEPATH += $$quote(T:/_outdir/libtorrent/libtorrent64_gcc/include)
 #Point this to the zlib include folder(libtorrent's if you used that)
-INCLUDEPATH += $$quote(C:/qBittorrent/RC_0_15/zlib)
+INCLUDEPATH += $$quote(T:/_outdir/Zlib/Zlib64_gcc/include)
 #Point this to the openssl include folder
-INCLUDEPATH += $$quote(C:/OpenSSL/include)
+INCLUDEPATH += $$quote(T:/_outdir/OpenSSL/OpenSSL64_gcc/include)
 
 #Point this to the openssl lib folder
-LIBS += $$quote(-LC:/OpenSSL/lib/VC)
+LIBS += $$quote(-LT:/_outdir/OpenSSL/OpenSSL64_gcc/lib)
 #Point this to the libtorrent lib folder
-LIBS += $$quote(-LC:/qBittorrent/RC_0_15/bin/<path-according-to-the-build-options-chosen>)
+LIBS += $$quote(-LT:/_outdir/libtorrent/libtorrent64_gcc/lib)
 #Point this to the boost lib folder
-LIBS += $$quote(-LC:/qBittorrent/boost_1_46_0/stage/lib)
+LIBS += $$quote(-LT:/_outdir/Boost/Boost64_gcc/lib)
+LIBS += $$quote(-LT:/_outdir/Zlib/Zlib64_gcc/lib)
 
 # LIBTORRENT DEFINES
 DEFINES += BOOST_ALL_NO_LIB
+DEFINES += BOOST_ASIO_ENABLE_CANCELIO
 DEFINES += BOOST_ASIO_HASH_MAP_BUCKETS=1021
+DEFINES += BOOST_ASIO_SEPARATE_COMPILATION
 DEFINES += BOOST_EXCEPTION_DISABLE
 DEFINES += BOOST_SYSTEM_STATIC_LINK=1
-DEFINES += BOOST_THREAD_USE_LIB
-DEFINES += BOOST_THREAD_USE_LIB=1
 DEFINES += TORRENT_USE_OPENSSL
+DEFINES += TORRENT_LINKING_SHARED
+DEFINES += TORRENT_EXPORT_EXTRA
 DEFINES += UNICODE
 DEFINES += WIN32
 DEFINES += WIN32_LEAN_AND_MEAN
@@ -31,8 +34,9 @@
 DEFINES += _FILE_OFFSET_BITS=64
 DEFINES += _SCL_SECURE_NO_DEPRECATE
 DEFINES += _UNICODE
+DEFINES += UNICODE
 DEFINES += _WIN32
-DEFINES += _WIN32_WINNT=0x0500
+DEFINES += _WIN32_WINNT=0x0601
 DEFINES += _WIN32_IE=0x0500
 DEFINES += __USE_W32_SOCKETS
 DEFINES += WITH_SHIPPED_GEOIP_H

Change "DEFINES += _WIN32_WINNT=0x0601" according to windows-version=[2k|xp|vista|win7] config option of libtorrent or leave it alone, if this option was not given when configuring libtorrent.

[Dayman]

Ok. It seems that either libtorrent can't handle over 150 inactive (but nonpaused) torrents or i'm just giving up on this case.

EDIT:
Is this real life? No active uTP connections, no DHT nodes. Router does it's job (see attachments for 'tcpdump -p -n -v'): packets go to ppp0 and get routed to br0, packets' ids match :-/
Yeah, it's either windows or libtorrent.

[pragmaticenigma]

I'm not sure why you are trying to compile 64bit when the author doesn't appear to have an 64bit code.

Have you been using your own compiled 64bit version of qbittorrent this whole time?

[pragmaticenigma]

150+ torrents? All versions of Windows (to the best of my knowledge) have a limited number of available concurrent connections.  Your likely exceeding this limit and since DHT is UDP the packets are being dropped without a trace.

I don't know how your testing this on your VMs (just using one torrent or using the same number of torrents.) but what happens if you just activate 1 of your torrents?