Page 3 of 6

Change in source tree structure in hosting site and libtorrent new release.

Posted: Tue Jun 24, 2014 2:59 am
by littletree76

On 20-June-2014, source tree structure in qBittorrent hosting site has been revamped and previous version of qBittorrent build script build-qbittorrent no longer work completely. I have updated URL address for checking out truck branch (master copy) of qBittorrent source tree as follows:

Old:    https://github.com/qbittorrent/qBittorr ... hes/master
New:  https://github.com/qbittorrent/qBittorrent/truck

The change in source tree structure only affect checking out of truck branch (alpha version 3.2.0 under active development) but not any release version (3.1.9.2). Change in source tree structure rarely happen and any developer should be able to provide proper URL address for checkout operation. Updated new version of build script build-qbittorrent has been uploaded to Dropbox site.

Latest release version of libtorrent Unix package has also been upgraded from version 0.16.16 to version 0.16.17 and alpha version has been upgrade to 1.0.0 RC3. I have recompiled libtorrent Unix package on my iMac with the build script build-libtorrent as follows. As for the alpha version 1.0.0 RC 3 of libtorrent, I have attempted compilation with this version few weeks ago and encountered undefined symbol errors, thus not until source code of qBittorrent application has catched up with this version of libtorrent there is no point of trying to compile with this alpha version (same approach applied to latest Qt version 5.3.0).

build-libtorrent -t 0.16.17

All disk image files (qbittorrent-3.1.9.2.dmg for release version and qbittorrent-master.dmg for alpha version 3.2.0) have been rebuilt with the new libtorrent library and uploaded to the Dropbox site. They have been installed and tested successfully on my iMac with test.torrent in the Dropbox site. All versions of qBittorrent application are built with following versions of libraries:

Qt 4.8.6
Libtorrent 0.16.17.0
Boost 1.55.0

All updated files are ready for access at usual Dropbox site: https://www.dropbox.com/sh/45qkagvurvxh3tp/UX0Ni-BVK3

I can't emphasise enough the important of tweaking parameters in qBittorrent application to avoid hangup (it happened to me with version 0.16.16 of libtorrent), particularly under following sections. These settings are for fibre broadband Internet connection with download bandwidth of 100 Mbps and upload bandwidth of 50 Mbps, adjust according to your Internet bandwidths. So make adjustments on these parameters before making complain. Note using a separate port for DHT causes downloading stage to start much earlier after torrent has been added.

I have been using alpha version 3.2.0 of qBittorrent application on my iMac (2.8 GHz Core-i7, 16 GB DDR3 RAM) with satisfactory result (better than uTorrent 3.1.9 which has been discarded few months ago).

[Connection]

Port used for incoming connection:                          54xxx
Use UPnP/NAT-PMP port forwarding for my router:  not checked
Use different port on each startup:                            not checked
(Make sure UPnP or NAT-PMP are disabled and specific port for incoming connections is opened on your router. Download bandwidth really suffers without any opened port.)

Global maximum number of connections:        600
Maximum number of connections per torrent:  80
Global maximum number of upload slots:        150
Maximum number of upload slots per torrent:  20

[Speed]

Upload:      4800 KiB/s (note Kilo-Byte-per-Second)
Download:  7200 KiB/s

[BitTorrent]

Enable DHT to find more peers:                    checked
Use a different port for DHT and BitTorrent:  checked
DHT port:                                                      68xx

Encryption mode:  Require encryption
(Ensure your ISP unable to block bittorrent traffic by analysing content of package.)

[Advanced]

Transfer list refresh interval:                              3000 ms
Maximum number of half-opened connections:  80
Exchange trackers with other peers:                  checked
Always announce to all trackers:                        checked

Release of Mac OS 10.9.4

Posted: Wed Jul 02, 2014 2:17 am
by littletree76

Apple has just released new version 10.9.4 of Mac OS X (probably the last one before launch of Mac OS 10.10 Yosemite). I updated Mac OS X on my iMac and recompiled libtorrent Unix package and qBittorrent application as follows:

build-libtorrent -t 0.16.17  (build-libtorrent -t master for version 1.0 RC3 does not compile properly with existing source code of qBittorrent)
build-qbittorrent -t 3.1.9.2
build-qbittorrent -t master (for alpha version 3.2.0)

Note the two build scripts check out source code from online source depositories according to version given in command line. Thus the source codes are always the latest (particularly true for alpha versions still under active development) and not relied on some outdated compressed archive files.

Updated disk image files have been uploaded to Dropbox site for free access. Everything work fine in new Mac OS 10.9.4, in fact most operations in qBittorrent application have been speeded up. I am not sure whether the speedup is caused by new Mac OS 10.9.4 or the effect of removing software firewall Intego NetBarrier from Mac OS X. Since I already have hardware firewall set up in my wireless router, there is no point of cascading network traffic through yet another barrier of software firewall.

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Wed Jul 02, 2014 8:37 pm
by Nemo
Thanks littletree76, great work :).

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Sat Jul 05, 2014 7:39 pm
by sledgehammer_999
@littletree76

Just so you know, libtorrent 1.0.0 support is only available on git master. Are you testing with it? On my machine it compiles fine.
If master fails, please post compilation logs so I can see what is happening.

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 07, 2014 11:46 pm
by littletree76

@sledgehammer_999

When I mentioned failure in compilation , it meant the latest trunks (master copys) of source trees for both libtorrent and gbittorrent do not work properly together few weeks ago. Then I could compile any version of libtorrent Unix package through my build script build-libtorrent without issue in older Mac OS 10.9.3, just specify the required version as command line argument as follows (argument "master" for trunk of source tree for both build scripts). In both build scripts, all source codes are checked out directly from source depository through svn client without using any compressed archive file, hence they are always the latest versions still under development at the time of script execution.

build-libtorrent -t master
-- for trunk version 1.0.0-RC3 of libtorrent source tree at: svn://svn.code.sf.net/p/libtorrent/code/trunk

Once the libtorrent library has been compiled and installed successfully under default /usr/local directory in Mac OS 10.9.4, I use another build script build-qbittorrent to compile and build qbittorrent application with the newly installed libtorrent library, that was when error messages appeared during make process and the process was halted without completion.

build-qbittorrent -t master
-- for trunk version 3.2.0 of qbittorrent source tree at: https://github.com/qbittorrent/qBittorrent/trunk

Today I have just executed the build-libtorrent build script again to compile trunk version (1.1.0) of libtorrent package and it failed, so I reverted back to older release version 1.0.0 (released on  2-Jul-2014) and compilation still failed. So I reverted back yet older release versions 1.0.0-RC3 (released on 23-Jun-2014) and 1.0.0-RC (released on 8-May-2014), surprisingly both compilations failed this time (in new Mac OS 10.9.4). Please find four make log files generated during compilations of libtorrent package attached with this post.

build-libtorrent -t master (alpha version 1.1.0)
build-libtorrent -t 1.0
build-libtorrent -t 1.0-rc3
build-libtorrent -t 1.0-rc

Failure occurred at the same spot (file ciso646 not found) during the make process for versions 1.0.0, 1.0.0-RC3 and 1.0.0-RC whereas make process never start properly at all for latest trunk version 1.1.0.

  CC      ../ed25519/src/add_scalar.lo
In file included from ../ed25519/src/add_scalar.c:1:
In file included from ../ed25519/src/ed25519.h:6:
In file included from ../include/libtorrent/config.hpp:55:
In file included from /opt/local/include/boost/config.hpp:35:
/opt/local/include/boost/config/compiler/clang.hpp:175:10: fatal error: 'ciso646' file not found
#include <ciso646>
        ^
1 error generated.
make[1]: *** [../ed25519/src/add_scalar.lo] Error 1

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Tue Jul 08, 2014 5:44 am
by sledgehammer_999
I can't review the logs atm but just so you know libtorrent 1.0.0 now resides here: svn://svn.code.sf.net/p/libtorrent/code/branches/RC_1_0/

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Tue Jul 08, 2014 7:43 pm
by sledgehammer_999
Try the latest from the RC_1_0 branch. If it still fails please share here:
1. Your libtorrent configuration options
2. The output of make
3. Boost version used.

I'll to forward them to the libtorrent author.

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Thu Jul 10, 2014 3:18 am
by littletree76

Previously my build script build-libtorrent for building libtorrent check out source code through tags instead of branches directory of libtorrent source tree. But I have tried both source directories and both compilations failed in latest Mac OS 10.9.4 for the same cause (file ciso646 not found). From now on I will only check out source code through branches instead of tags directory for both libtorrent and qbittorrent. Please find configure.log, config.log and make log files attached with this post for your reference.

The version of boost package in use is 1.55.0_6 which is the latest installed through MacPorts 2.3.1 package manager. As for configuration before compilation, following two lines are taken from my build script build-libtorrent:
autotool.sh
configure --enable-debug=no

Followings are log of compilation process and last 30 lines of make log file extracted from display of Terminal application (similar to any Unix terminal) in Mac OS 10.9.4.

Code: Select all

Wangs-iMac:~/project/libtorrent$ ./build-libtorrent -b RC_1_0
You might be prompted for administrator password.

Checking out source code from svn repository at following address:
svn://svn.code.sf.net/p/libtorrent/code/branches/RC_1_0
Please be patient ......

Source code checked out successfully from svn repository.
Checked out revision 10097.

Executions of configuration and compilation started, please refer to log files for details.
Source directory: RC_1_0

+ cd RC_1_0
+ ./autotool.sh
+ ./configure --enable-debug=no
++ /usr/sbin/sysctl -n hw.physicalcpu
+ PHYSICAL_CORES=4
+ /usr/bin/make --silent clean
Making clean in include/libtorrent
Making clean in src
rm -f ./so_locations
Making clean in examples
Making clean in test
Making clean in bindings
Making clean in python
Making clean in tools
+ /usr/bin/make -j4
+ STATUS=2
+ set +x

Compilation failed, please refer to log file /Users/littletree/project/libtorrent/make.log.

Wangs-iMac:~/project/libtorrent$ tail -30 make.log
clang: warning: argument unused during compilation: '-I ../ed25519/src'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I /opt/local/include'
  CC       ../ed25519/src/add_scalar.lo
  CC       ../ed25519/src/fe.lo
In file included from ../ed25519/src/add_scalar.c:1:
In file included from ../ed25519/src/ed25519.h:6:
In file included from ../include/libtorrent/config.hpp:45:
In file included from /opt/local/include/boost/config.hpp:35:
/opt/local/include/boost/config/compiler/clang.hpp:175:10: fatal error: 'ciso646' file not found
#include <ciso646>
         ^
  CC       ../ed25519/src/ge.lo
1 error generated.
make[1]: *** [../ed25519/src/add_scalar.lo] Error 1
make[1]: *** Waiting for unfinished jobs....
clang: warning: argument unused during compilation: '-I .'
clang: warning: argument unused during compilation: '-I ../include'
clang: warning: argument unused during compilation: '-I ../ed25519/src'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I ../ed25519/src'
clang: warning: argument unused during compilation: '-I .'
clang: warning: argument unused during compilation: '-I ../include'
clang: warning: argument unused during compilation: '-I ../ed25519/src'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I /opt/local/include'
clang: warning: argument unused during compilation: '-I ../ed25519/src'
make: *** [all-recursive] Error 1

All versions of Unix packages installed through MacPorts package manger are shown below:

Code: Select all

Wangs-iMac:~/project/libtorrent$ port installed
The following ports are currently installed:
  apr @1.5.1_0 (active)
  apr-util @1.5.3_0 (active)
  autoconf @2.69_2 (active)
  automake @1.14.1_2 (active)
  boost @1.55.0_6+no_single+no_static+python27 (active)
  bzip2 @1.0.6_0 (active)
  ccache @3.1.9_0 (active)
  curl-ca-bundle @7.37.0_0 (active)
  cyrus-sasl2 @2.1.26_3+kerberos (active)
  db46 @4.6.21_9 (active)
  db_select @0.1_2 (active)
  dbus @1.8.6_0 (active)
  expat @2.1.0_0 (active)
  gdbm @1.11_0 (active)
  gettext @0.19.1_1 (active)
  icu @53.1_0 (active)
  jpeg @9a_1 (active)
  kerberos5 @1.11.3_0 (active)
  lcms @1.19_5 (active)
  libcomerr @1.42.9_0 (active)
  libedit @20121213-3.0_0 (active)
  libiconv @1.14_0 (active)
  libmagic @5.18_1 (active)
  libmng @1.0.10_3 (active)
  libpng @1.6.12_0 (active)
  libtool @2.4.2_3 (active)
  lynx @2.8.7rel.2_1+ssl (active)
  m4 @1.4.16_0 (active)
  ncurses @5.9_2 (active)
  openssl @1.0.1h_0 (active)
  perl5 @5.12.4_0+perl5_16 (active)
  perl5.16 @5.16.3_0 (active)
  pkgconfig @0.28_0 (active)
  python27 @2.7.8_0 (active)
  python_select @0.3_3 (active)
  qt4-mac @4.8.6_0 (active)
  serf1 @1.3.4_0 (active)
  sqlite3 @3.8.4.3_0 (active)
  subversion @1.8.9_0 (active)
  tiff @4.0.3_4 (active)
  xz @5.0.5_0 (active)
  zlib @1.2.8_0 (active)

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 12:20 am
by sledgehammer_999
Reported this upstream. Hopefully arvid will be able to fix it. bug report: https://code.google.com/p/libtorrent/is ... ail?id=648

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 12:31 am
by sledgehammer_999
Until arvid responds AND if you have time, retry a build of the latest RC_1_0. I see that he has pushed changes on how ed25519 is being built, so it might have fixed your problem.

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 6:48 am
by littletree76

Yes I have noticed changes in branch RC_1_0 of source tree as well. I used my build script build-libtorrent to check out source from that branch RC_1_0 (version 1.0.1) and libtorrent package compiled successfully this time without issue.

build-libtorrent -b RC_1_0

But when I rebuild both release 3.1.9.2 and master copy (alpha version 3.2.0) of qbittorrent (checked out source code directly from tag release-3.1.9.2 and master of source tree) with this new release of libtorrent package, both compilations failed. Please find log files for make processes of libtorrent 1.0.1, qbittorrent 3.1.9.2 and qbittorrent 3.2.0 attached with this post.

build-qbittorrent -t 3.1.9.2
build-qbittorrent -t master

Note in both of my build scripts, I have switched from checking out source code from tags directories to branches directories, unfortunately compilation for qbittorrent 3.1.9.2 failed with source code checked out from branch v3_1_x, so I have to revert back to checking out source code from tag release-3.1.9.2 (though no such problem with master version 3.2.0 of qbittorrent source tree). Probably by comparing source code between following two locations should be able to pinpoint differences between tag release-3.1.9.2 and branch v3_1_x. I use svn client to check out all source code in both build scripts since it is compatible with github source depository.

https://github.com/qbittorrent/qBittorr ... se-3.1.9.2
https://github.com/qbittorrent/qBittorr ... hes/v3_1_x

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 12:42 pm
by sledgehammer_999
v3_1_x: The errors are due to using libtorrent 1.0.x. That branch doesn't have support for it. Only for 0.15.x and 0.16.x
master: That single error is totally weird. The code disables that function when libtorrent 1.0.0 is used. Are you sure you did a clean build of that branch and it didn't have leftovers from a build against libtorrent 0.16.x?

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 5:26 pm
by littletree76

I am very sure that before a new build for libtorrent is installed, all files related to old build of libtorrent are hunted down (through Unix find command) and removed from default installation directory /usr/local in Mac OS 10.9.4. In my build script build-libtorrent for building all versions of libtorrent Unix package, you should find following two lines at the end of the script:

find -d /usr/local -iname '*libtorrent*' -exec $rm -rf {} \; -ls
sudo make -j1 install

Even without the first line to remove all existing files of old build, the next line where "make install" is executed should over-write existing files in default installation directory (/usr/local). When building master version (3.2.0) of qbittorrent with new macxconf.pri file provided by you sometime ago (autotools are in use), following error messages always appear in configure log file:

configure: Running qmake to generate the makefile...
/Users/littletree/project/qbittorrent/qBittorrent/trunk/macxconf.pri:7: Unknown test function: sts

Which means following section of configuration instructions have been invalid and bypassed all the time during the configure process:

sts($$OUT_PWD/../conf.pri) {
    include($$OUT_PWD/../conf.pri)
} else {
    include(conf.pri)
}

Can this be the cause of failure in compiling qbittorrent 3.2.0 application with new version 1.0.1 of libtorrent package ? Please find two log files generated during the configure process attached with this post.

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Mon Jul 14, 2014 6:45 pm
by sledgehammer_999
I have successfully built 3.2.0 with libtorrent 1.0.1 on Windows.

You delete the files from the final destination but what about the build directory? Is it cleaned?

You seem to either have copied the maxconf.pri incompletely or I provided an incomplete version of it. "sts($$OUT_PWD/../conf.pri) {" should be "exists($$OUT_PWD/../conf.pri) {"

Re: Compiling latest qBittorrent with MacPorts package manager

Posted: Tue Jul 15, 2014 3:43 am
by littletree76

All files related to libtorrent under default third-party software install directory /usr/local are cleared before new versions of the files are installed. Similarly build directories where both of my build scripts build-libtorrent and build-qbittorrent are located are always cleared completely before executions of the scripts to build new versions. Thus follows are content of both build directories before executions of both scripts:

~/project/qbittorrent:
test.torrent
macxconf.pri.autotools
build-qbittorrent

~/project/libtorrent:
build-libtorrent

I am able to do that because all source files are always checked out from online source depository in real time according to version specified on command line when the build scripts are executed. Even the GeoIP.dat data file is downloaded in real time in the build script. Thus no compressed archive files or any other files are needed before executions of the scripts since source files are not handled independently but all of them are always checked out in one go. Another advantage is I never have to worry about dependencies among these source files.

So I disable use of compiler cache (through Unix ccache command) in both scripts, edit macxconf.pri configuration file to correct the typo error and rebuild both libtorrent package (version 1.0.1 checked out from branch RC_1_0) and qbittorrent application (version 3.2.0 checked out from trunk directory). The Unix package libtorrent 1.0.1 was built successfully whereas same error happened again at the same spot during make process for qbittorrent 3.2.0 application. Please find a log file generated during make process of qbittorrent attached with this post.

Judge by error message, probably there is conflict in member setDHTPort of data structure QBtSession between qbittorrent application and libtorrent package.

Code: Select all

Wangs-iMac:~/project/qbittorrent$ tail -8 qbittorrent-make-master.log
/usr/bin/clang++ -c -pipe -DTORRENT_USE_OPENSSL -DWITH_SHIPPED_GEOIP_H -DBOOST_ASIO_HASH_MAP_BUCKETS=1021 -DBOOST_EXCEPTION_DISABLE -DBOOST_ASIO_ENABLE_CANCELIO -DBOOST_ASIO_DYN_LINK -DTORRENT_LINKING_SHARED -I/usr/local/include -I/usr/local/include/libtorrent -O2 -arch x86_64 -Xarch_x86_64 -mmacosx-version-min=10.9 -Wall -W -DWITH_GEOIP_EMBEDDED -DQT_NO_DEBUG_OUTPUT -DVERSION=\"v3.2.0alpha\" -DVERSION_MAJOR=3 -DVERSION_MINOR=2 -DVERSION_BUGFIX=0 -DQT_NO_CAST_TO_ASCII -DQT_USE_FAST_CONCATENATION -DQT_USE_FAST_OPERATOR_PLUS -DBOOST_FILESYSTEM_VERSION=2 -DQT_NO_DEBUG -DQT_XML_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -DQT_SHARED -I/opt/local/share/qt4/mkspecs/macx-g++ -I. -I. -I/opt/local/Library/Frameworks/QtCore.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtNetwork.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtGui.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtXml.framework/Versions/4/Headers -I. -Iqtsingleapp -Iqtlibtorrent -Iwebui -Iwebui/qjson -Itracker -Ipreferences -Ilineedit/src -Iproperties -Isearchengine -Irss -Itorrentcreator -Igeoip -Ipowermanagement -I/opt/local/Library/Frameworks/QtXml.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtGui.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtNetwork.framework/Versions/4/Headers -I/opt/local/Library/Frameworks/QtCore.framework/Versions/4/Headers -I/opt/local/include -I. -F/opt/local/Library/Frameworks -F/opt/local/lib -o moc_alertdispatcher.o moc_alertdispatcher.cpp
moc_qbtsession.cpp:361:22: error: no member named 'setDHTPort' in 'QBtSession'
        case 68: _t->setDHTPort((*reinterpret_cast< int(*)>(_a[1]))); break;
                 ~~  ^
1 error generated.
make[1]: *** [moc_qbtsession.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [sub-src-make_default] Error 2