Windows tool for migration from Bittorrent/uTorrent/Deluge to qBittorrent

Windows specific questions, problems.
Post Reply
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Windows tool for migration from Bittorrent/uTorrent/Deluge to qBittorrent

Post by rumanzo »

Disappointed in the existing application on the ruby, which could not handle all my torrents, I wrote my own application on golang.
I tested it on my own torrent database (3k+) and test torrents on Bittorrent/uTorrent clients (version 3+).
Features:
  • Processing all torrents
  • Processing torrents with subdirectories and without subdirectories
  • Processing torrents with renamed files
  • Processing torrents with non-standard encodings (for example, cp1251)
  • Processing of torrents in the not ready state *
  • Save date, metrics, status. **
  • Import of tags and labels
  • Multithreading
  • Path replace mechanism
* This torrents will not be done (0%) and will need force rehash
* If you migrate from windows to linux and use replace function attention that multiple flags -r processing one by one
* If you migrate from windows to linux and yours torrent files saves to some place you must use flag --search with actual paths in yours system
* If you migrate from windows to linux you may need to define path separathor with --sep flag
** The calculation of the completed parts is based only on the priority of the files in torrent
*** Partially downloaded torrents will be visible as 100% completed, but in fact you will need to do a rehash. Without rehash torrents not will be valid. This is due to the fact that conversion of .dat files in which parts of objects are stored is not implemented.

Don't forget before use make backup bittorrent\utorrent, qbittorrent folder. and config %APPDATA%/Roaming/qBittorrent/qBittorrent.ini. Close all this program before.

Usage:
If you MacOs user or Linux user don't forget to make the file executable:

Code: Select all

$ chmod +x bt2qbt_v1.3_amd64_macos
1.) Help (from cmd or powerwhell)

Code: Select all

 C:\Users\user\Downloads\bt2qbt_v1.3_amd64.exe [OPTIONS]

Application Options:
  -s, --source=         Source directory that contains resume.dat and torrents
                        files (default:
                        C:\Users\user\AppData\Roaming\uTorrent\)
  -d, --destination=    Destination directory BT_backup (as default) (default:
                        C:\Users\user\AppData\Local\qBittorrent\BT_backup\)
  -c, --config=         Source directory that contains resume.dat and torrents
                        files (default:
                        C:\Users\user\AppData\Roaming\qBittorrent\qBittorren-

                        t.ini)
      --without-labels  Do not export/import labels
      --without-tags    Do not export/import tags
  -t, --search=         Additional search path for torrents files
                        Example: --search='/mnt/olddisk/savedtorrents'
                        --search='/mnt/olddisk/workstorrents'
  -r, --replace=        Replace paths.
                        Delimiter for from/to is comma - ,
                        Example: -r "D:\films,/home/user/films" -r
                        "D:\music,/home/user/music"

      --sep=            Default path separator that will use in all paths. You
                        may need use this flag if you migrating from windows to
                        linux in some cases (default: \)

Help Options:
  -h, --help            Show this help message
2.) If you just run application, it will processing torrents from %APPDATA%\uTorrent\ to %LOCALAPPDATA%\qBittorrent\BT_BACKUP\
Example:

Code: Select all

It will be performed processing from directory C:\Users\user\AppData\Roaming\uTorrent\ to directory C:\Users\user\AppData\Local\qBittorrent\BT_backup\
Check that the qBittorrent is turned off and the directory C:\Users\user\AppData\Local\qBittorrent\BT_backup\ and config C:\Users\user\AppData\Roaming\qBittorrent\qBittorrent.ini is backed up.


Press Enter to start

Started
1/2 Sucessfully imported 1.torrent
2/2 Sucessfully imported 2.torrent

Press Enter to exit
3.) Run application from cmd or powershell with keys, if you want change source dir or destination dir, or export/import behavior
Example:

Code: Select all

C:\Users\user\Downloads> .\bt2qbt_v1.3_amd64.exe -s C:\Users\user\AppData\Roaming\BitTorrent\
It will be performed processing from directory C:\Users\user\AppData\Roaming\BitTorrent\ to directory C:\Users\user\AppData\Local\qBittorrent\BT_backup\
Check that the qBittorrent is turned off and the directory C:\Users\user\AppData\Local\qBittorrent\BT_backup\ is backed up.

Press Enter to start
Started
1/3233 Sucessfully imported 1.torrent
2/3233 Sucessfully imported 2.torrent
3/3233 Sucessfully imported 3.torrent
...
3231/3233 Sucessfully imported 3231.torrent
3232/3233 Sucessfully imported 3232.torrent
3233/3233 Sucessfully imported 3233.torrent

Press Enter to exit
Known issuses:
  • Unknown
Here's the project: https://github.com/rumanzo/bt2qbt
Code:  https://github.com/rumanzo/bt2qbt/blob/master/bt2qbt.go
Releases: https://github.com/rumanzo/bt2qbt/releases

Deluge branch:
Here's the project: https://github.com/rumanzo/deluge2qbt
code: https://github.com/rumanzo/deluge2qbt/b ... uge2qbt.go
Releases: https://github.com/rumanzo/deluge2qbt/releases

If there are any errors or if it help you - write there
Last edited by rumanzo on Wed Aug 14, 2019 8:32 pm, edited 1 time in total.
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

[quote="rumanzo"]
  • Processing torrents with non-standard encodings (for example, cp1251)
[/quote]

Do you mean filename encodings inside the torrent file? If so, what do you do with that?
.torrent files should have utf8 encoding for the filenames. Otherwise, libtorrent replaces invalid utf8 codepoints with an underscore "_". Do you handle this?
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

[quote="sledgehammer_999"]
[quote="rumanzo"]
  • Processing torrents with non-standard encodings (for example, cp1251)
[/quote]

Do you mean filename encodings inside the torrent file? If so, what do you do with that?
.torrent files should have utf8 encoding for the filenames. Otherwise, libtorrent replaces invalid utf8 codepoints with an underscore "_". Do you handle this?
[/quote]
To be honest, I just take the name field name.utf-8 and path.utf-8 in the torrent file if they exists, and in this form I feed to the bencode library. I did not meet any problems, and I did not think that complications could occur. But I can do a check on the validity of the runes (https://golang.org/pkg/unicode/utf8/#Valid).

Despite the fact that the code works nominally, for me there are still things that I do not really understand where to get it  and I would like to rewrite it. But I'm not at all familiar with c/c++ code. If you could tell me a couple of things - that would be fine.
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

[quote="rumanzo"]
Despite the fact that the code works nominally, for me there are still things that I do not really understand where to get it  and I would like to rewrite it. But I'm not at all familiar with c/c++ code. If you could tell me a couple of things - that would be fine.
[/quote]

Ask here, and if I see it I'll try to answer.
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

[quote="sledgehammer_999"]
[quote="rumanzo"]
Despite the fact that the code works nominally, for me there are still things that I do not really understand where to get it  and I would like to rewrite it. But I'm not at all familiar with c/c++ code. If you could tell me a couple of things - that would be fine.
[/quote]

Ask here, and if I see it I'll try to answer.
[/quote]
1.) Will the check runes for utf-8 be correct?
2.) I do not understand how to correctly calculate "blocks per piece" in fastresume. Now I'm using the logic from the previous scripts (on ruby), it works, but I do not like the concept of using fields from BitTorrent resume.dat for this. How to count this field?
3.) I fill the "pieces" field by the number of torrentfile["info"]["lenght"] or sum lenght of files if it multifile torrent / torrentfile["info"]["piece length"] rounded to a larger number. Is it correct?
4.) Now i rehash all my torrents, and some my torrents in which not all files have been selected for downloading, no fully completed. That's ok, bittorrent/utorent create ~BitTorrentPartFile.*.dat, which contain some data, and i not convert it. I am confused that after rehash torrents just stuck. Any idea why this is possible?
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

[quote="rumanzo"]
1.) Will the check runes for utf-8 be correct?
[/quote]

IIRC, normally you don't need to bother with the filenames. You do that if a) the user has renamed the file from inside utorrent b)or has explicitly moved it outside of the folder
In any case, you can't edit the .torrent files because it will change the infohash. The original filenames come from the .torrent. Libtorrent does a sanity check, and replaces invalid utf8 codepoints with and underscore ("_"). And uses this new sanitized string to look for files on disk.

I hope we are talking about the same thing/spot and I am not confusing you.

[quote="rumanzo"]
2.) I do not understand how to correctly calculate "blocks per piece" in fastresume. Now I'm using the logic from the previous scripts (on ruby), it works, but I do not like the concept of using fields from BitTorrent resume.dat for this. How to count this field?
[/quote]

I hope I talking about the same thing.
Unless I am mistaken blocks are always 16KiB. So a piece contains set number of 16KiB blocks. The only exception is the end piece which might contain less data.
I assume that you have to take the piece size from the .torrent file and divide it by 16KiB.

[quote="rumanzo"]
3.) I fill the "pieces" field by the number of torrentfile["info"]["lenght"] or sum lenght of files if it multifile torrent / torrentfile["info"]["piece length"] rounded to a larger number. Is it correct?
[/quote]

I think not. Maybe the calculation of the number of pieces is correct. The data you fill the piece array might not be correct.
Some more info:
Here's an old explanation by me on what I had found on resume.dat when I experimented with it. Link to specific comment. You might want to read the thread from the start to get any other useful info.
And from libtorrent docs here's an explanation on what various fields in the fastresume do: https://libtorrent.org/manual-ref.html#fast-resume

[quote="rumanzo"]
4.) Now i rehash all my torrents, and some my torrents in which not all files have been selected for downloading, no fully completed. That's ok, bittorrent/utorent create ~BitTorrentPartFile.*.dat, which contain some data, and i not convert it. I am confused that after rehash torrents just stuck. Any idea why this is possible?
[/quote]

Maybe it is in weird state, or you just don't have any peers anymore?
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

IIRC, normally you don't need to bother with the filenames. You do that if a) the user has renamed the file from inside utorrent b)or has explicitly moved it outside of the folder
In any case, you can't edit the .torrent files because it will change the infohash. The original filenames come from the .torrent. Libtorrent does a sanity check, and replaces invalid utf8 codepoints with and underscore ("_"). And uses this new sanitized string to look for files on disk.

I hope we are talking about the same thing/spot and I am not confusing you.
It looks like I do not really need to worry about this.
I hope I talking about the same thing.
Unless I am mistaken blocks are always 16KiB. So a piece contains set number of 16KiB blocks. The only exception is the end piece which might contain less data.
I assume that you have to take the piece size from the .torrent file and divide it by 16KiB.
It is! So simple, thanks.
I think not. Maybe the calculation of the number of pieces is correct. The data you fill the piece array might not be correct.
Some more info:
Here's an old explanation by me on what I had found on resume.dat when I experimented with it. Link to specific comment. You might want to read the thread from the start to get any other useful info.

And from libtorrent docs here's an explanation on what various fields in the fastresume do: https://libtorrent.org/manual-ref.html#fast-resume
Thanks for links. But I'm not sure that we're talking about the same thing. I mean the length of the "pieces" field. I fill it with 0 or 1 (0x00 or 0x01), it's correct. If calculation of the number of pieces is correct, it's all right.
Maybe it is in weird state, or you just don't have any peers anymore?
This does not apply to all torrents, and in the log, I see a io error ... error: permission denied. Apparently this is due not to the torrents themselves, but to something else. I'll try to figure out the other day.


P.S. I try to modify top message and got error:
Forbidden
You don't have permission to access /index.php on this server.
Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.
Last edited by rumanzo on Sun May 06, 2018 2:22 pm, edited 1 time in total.
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

[quote="rumanzo"]
Thanks for links. But I'm not sure that we're talking about the same thing. I mean the length of the "pieces" field. I fill it with 0 or 1 (0x00 or 0x01), it's correct. If calculation of the number of pieces is correct, it's all right.
[/quote]

IMO, the length should be taken from the .torrent file. It has a "pieces" field too.
About the "pieces" field in the fastresume: That's a byte array (or byte string). Each element is one byte(character) and represents one piece. Each byte has 8bits. Setting the 1st bit means we have that piece. Setting the 2nd means that we verified that piece aka it passed the hash check. If that piece is unverified, it usually gets hash-checked during seeding when it is requested for upload.
So, if I am not mistaken the correct values are:
1. 0x0 -> piece not downloaded
2. 0x80 -> piece downloaded but not verified
3. 0xc0 -> piece downloaded and verified
4. 0x40 -> I don't think this is valid. It means piece not downloaded but verified.

(those values were obtained from online binary to hex converters)

[quote="rumanzo"]
This does not apply to all torrents, and in the log, I see a io error ... error: permission denied. Apparently this is due not to the torrents themselves, but to something else. I'll try to figure out the other day.
[/quote]

Then that probably means that something's wrong with the conversion.
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

[quote="sledgehammer_999"]
IMO, the length should be taken from the .torrent file. It has a "pieces" field too.
[/quote]
It has, but I can not find the connection between that number and what should be.
For example:
In torrent file - piece lenght = 8388608, len pieces - 105960
In fastresume file - len pieces - 5298.
It's seems like len(fr[pieces]) = len(tr[pieces])/20. But i don't sure and don't understand why.
[quote="sledgehammer_999"]
About the "pieces" field in the fastresume: That's a byte array (or byte string). Each element is one byte(character) and represents one piece. Each byte has 8bits. Setting the 1st bit means we have that piece. Setting the 2nd means that we verified that piece aka it passed the hash check. If that piece is unverified, it usually gets hash-checked during seeding when it is requested for upload.
[/quote]
Correct.

[quote="sledgehammer_999"]
Then that probably means that something's wrong with the conversion.
[/quote]
It's sad. Will debug this.
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

rumanzo wrote: It has, but I can not find the connection between that number and what should be.
For example:
In torrent file - piece lenght = 8388608, len pieces - 105960
In fastresume file - len pieces - 5298.
It's seems like len(fr[pieces]) = len(tr[pieces])/20. But i don't sure and don't understand why.
It seems I was slightly wrong.
The bittorrent BEP says this about the "pieces" field:
pieces maps to a string whose length is a multiple of 20. It is to be subdivided into strings of length 20, each of which is the SHA1 hash of the piece at the corresponding index.
And that's why you need to divide by 20 in your example.
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

sledgehammer_999 wrote:
rumanzo wrote: It has, but I can not find the connection between that number and what should be.
For example:
In torrent file - piece lenght = 8388608, len pieces - 105960
In fastresume file - len pieces - 5298.
It's seems like len(fr[pieces]) = len(tr[pieces])/20. But i don't sure and don't understand why.
It seems I was slightly wrong.
The bittorrent BEP says this about the "pieces" field:
pieces maps to a string whose length is a multiple of 20. It is to be subdivided into strings of length 20, each of which is the SHA1 hash of the piece at the corresponding index.
And that's why you need to divide by 20 in your example.
Thanks!

And I found problem with my torrents - why then on some files there was an attribute Read-only.
And I still can't modify top message, who can help with that?
Forbidden
You don't have permission to access /index.php on this server.
Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.
sledgehammer_999
Administrator
Administrator
Posts: 2416
Joined: Sun Jan 23, 2011 1:17 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by sledgehammer_999 »

Do you click the "Modify" button in your first post and you have those errors?
rumanzo
VIP
VIP
Posts: 95
Joined: Fri Dec 29, 2017 6:46 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by rumanzo »

[quote="sledgehammer_999"]
Do you click the "Modify" button in your first post and you have those errors?
[/quote]
Yes, if i try modify throw top button "Modify" a get this error. If i try use down button, nothing happens.
User avatar
Peter
Administrator
Administrator
Posts: 1915
Joined: Wed Jul 07, 2010 6:14 pm

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by Peter »

[quote="rumanzo"]
[quote="sledgehammer_999"]
Do you click the "Modify" button in your first post and you have those errors?
[/quote]
Yes, if i try modify throw top button "Modify" a get this error. If i try use down button, nothing happens.
[/quote]

Can you please PM me the error you get when you try Modify?
Thank you!
- qBittorrent team - server and forum administrator.
- Hungarian translation reviewer/moderator (+ translator).

Join the official qBittorrent Discord!
https://discord.gg/ma66Vv4
Gun
Newbie
Newbie
Posts: 2
Joined: Tue May 08, 2018 12:44 am

Re: Windows CLI tool for migration from Bittorrent/uTorrent (3.x) to qBittorrent

Post by Gun »

Hi, getting this error with 0.6 x64

Code: Select all

panic: interface conversion: interface {} is []interface {}, not string

goroutine 263 [running]:
main.(*NewTorrentStructure).gettrackers(0xc043cae2c0, 0xc042b47f00, 0x2, 0x4)
        C:/Users/rumanzo/eclipse-workspace/testinggo/src/bt2qbt/bt2qbt.go:212 +0x1e9
main.logic(0xc042b31120, 0x1e, 0xc042a590e0, 0xc04204c2f0, 0xc04200e180, 0xc04200e181, 0xc04204c300, 0xc042c6a2a0, 0x1af, 0xc04207dc90, ...)
        C:/Users/rumanzo/eclipse-workspace/testinggo/src/bt2qbt/bt2qbt.go:407 +0xa79
created by main.main
        C:/Users/rumanzo/eclipse-workspace/testinggo/src/bt2qbt/bt2qbt.go:510 +0xcd4
In 0.4 it looks like this

Code: Select all

panic: runtime error: index out of range

goroutine 275 [running]:
main.logic(0xc042d11630, 0x42, 0xc042cf3140, 0xc04204c300, 0xc04200e180, 0xc04200e181, 0xc04204c310, 0xc042c6fe00, 0x1af, 0xc04207dc90, ...)
        /home/kostinan/1T/testinggo/src/bt2qbt/bt2qbt.go:261 +0x5d40
created by main.main
        /home/kostinan/1T/testinggo/src/bt2qbt/bt2qbt.go:414 +0xcd4
Post Reply