Status of importing uTorrent Resume.dat?

[DeathStalker]

While I would love to try out QBT, I am not willing to migrate to another client (despite the issues with uT) until I am able to seemlessly import my current queue.  That is just TOO much of a pain.

Not being a coder, I am not clear on why this seems to be such a difficult task, when it is SO highly desired by SO many people?

Thanks.

[loki]

Mainly it would have to be "figured out" from scratch so to speak... or taken from closed-source (in this case utorrent) which 1. isn't allowed to be used in open-source software, 2. could be illegal...

So, keep wishing or do it yourself

Also check this topic is already discussed, http://qbforums.shiki.hu/index.php/topic,1562.0.html

[whatsgolden]

Importing a queue shouldn't be too much of a pain if they are waiting for download. A right click on a torrent in the list allows the magnet link to be copied.

Too many to copy? Use AutoHotkey and a macro.

Once you're done the entire magnet list can be pasted into qBittorrent.

The macro would only have to do:
rightclick(key next to R-CTRL)
downx3
enter(copy link)
alt-tab (to notepad)
paste
enter
alt-tab(back to utorrent)
down arrow
.. rinse and repeat .. It should take less than 5 minutes with the macro recorder.

[sledgehammer_999]

For seeding torrents you can already import them without the need of rechecking. Just use File->Import existing torrent... . Don't forget to tick the "skip data checking" checkbox.

[briantist]

I think I'm close to finishing up an importer. I was looking at a much earlier discussion here:
http://qbforums.shiki.hu/index.php/topic,1562.0.html

I liked the idea of using the .fastresume files because I wanted to preserve the added/completed times, ratio (uploaded and downloaded amounts), etc. So I've been able to generate what look like valid .fastresume files based on information in resume.dat and in the .torrent file itself.

The problem is that qBt is not using them at all. I've tried:

• Copying .fastresume into BT_backup, then add torrent in GUI
• Copying .fastresume and .torrent into BT_backup
• Adding torrent, closing qBt, replacing .fastresume, then reopening (it uses some of the info, like amount uploaded and downloaded, but ignores the times)

I am naming the files with the torrent's hash.

I suspect that the some of the info, like the times, are stored redundantly in qBittorrent-resume.ini, inside the torrents= line. It's a Byte array, and when I decode it and look at it in a hex editor, I can see that it's storing info about the torrents (their hash, where they're stored, the times, etc.).

But it looks like those files are saved as part of the qT framework, and that this is 'qSettings', and the Variant array is actually some kind of serialized object used internally by the framework. Unfortunately that makes it very difficult to parse from outside the application.

I am using Ruby, so the BEncoded files are no problem, and I was hoping to avoid the .ini stuff but unfortunately it seems to be clobbering what's in the .fastresume files somewhat.

If anyone has insight into how to best make use of a .fastresume file, or how to modify the INI from Ruby, it would be really helpful.

[sledgehammer_999]

I'll respond to your whole post tomorrow. Now I am too sleepy.
But here is a quick help for the .fastresume. Close qbt, copy .torrent + .fastresume in BT_backup, start qbt. It should pick them up. (Probably the save-path will be the default one though).

[briantist]

Thanks for the help.

I actually did try closing it, then adding both files to BT_backup. It acts rather strangely when I do that. The torrent is added (to the default download location), the uploaded, downloaded, and active time fields are correct. The added time is not correct (it uses the current time). The torrent is added in a state of "checking", but unlike with the other methods, it doesn't actually check the files. It doesn't even rename them to .!qB like it normally does when checking. Pausing and Starting don't trigger an actual check, it's just stuck in that state.

When I try the other methods, where I manually add the same .torrent file, it always checks all the files.

And actually that brings me to another question: I am filling in the pieces field (using the have field from resume.dat), so why is it checking the files all the time? Were you ever able to use a generated .fastresume to prevent the file checks like you wanted in your thread?

I have 5300+ torrents I want to import, so it would be useful to me, although honestly I wouldn't really mind the rechecks all that much (even if it does take a week).

[DeathStalker]

5300+???!!!  Ok, I thought *I* was bad with ~800!

LOL, I don't feel so bad now.  Though 88 of those are currently unfinished, and some are from magnetic links, so having to manually match the download path for them would be a ROYAL pain.

Glad to hear you're both making progress on things though!!! 

[briantist]

I like to use the list the in the torrent client as a convenient way to find and directly open files. Hence why I also care so much about the dates being correct.

What did you mean about having to manually match the download path for magnet links? I've only ever used a handful of magnet links.

[briantist]

Ok, a small update:

I fixed the problem I was having where closing qBt, putting both files in BT_backup resulted in a "stuck" torrent. I was incorrectly calculating 'blocks per piece' so that's fixed now.

But that method still doesn't result in the correct times, even though most of the other stats were used.

Fields in .fastresume that ARE being used:

• Uploaded
• Downloaded
• Time active
• UP Limit

Fields in .fastresume being IGNORED:

• Last Active
• Paused
• pieces/file sizes (times)

Regarding that last item, what I mean is that all the files are still checked. Whether I take the time from resume.dat (it does sometimes keep track of file times) or from each file's modified time, either way it checks all the files. I may need to do further testing with this, maybe with a larger torrent (maybe I'm just missing the checking).

Another thing I noticed: there is an inconsistency with where a torrent with multiple files is saved, depending on whether it is loaded from BT_backup or added to the client.

For example, consider a torrent with a name (name in info dict) of Files that contains File1.ext and File2.ext.
If your download directory is C:\Temp, and you add this torrent interactively, qB will automatically create C:\Temp\Files and download C:\Temp\Files\File1.ext and C:\Temp\Files\File2,ext. This is also the behavior you see with RSS downloads, and watched folders.

If you take this same torrent, and put it in BT_backup with qB closed, then reopen it, it will download the files in C:\Temp\File1.ext and C:\Temp\File2.ext.

The option to append the torrent caption has no effect on this behavior.

I could work around it in an importer, but right now my main challenge is still getting qB to recognize all fields of .fastresume (or finding some other way to import that data).

The ideal solution from an outside perspective would be if we could set all this through the web API. Add a torrent, set the download directory, modify torrent options, etc.

:sigh: I'm so close to SOMETHING here. I have all the data I could possibly need from uTorrent, just don't have a good way of getting it into qB.

[sledgehammer_999]

Is last_active(uTorrent) the same as active_time(libtorrent)? In qbt active_time is used for the "Time active:" label/column. If yes, do you write the value as a bencoded integer and not as a bencoded string?

pieces/file sizes (times) -> Maybe you are reconstructing wrong the pieces info in the .fastresume. Maybe you are off-by-one error. I think both indexes are zero based.

"Paused"-> When setting paused=1 try also setting auto_managed=0

Also from v3.1.0 onwards, qbt saves these custom fields in the .fastresume. Populate the ones you need. Strings are utf8.

• qBt-savePath (string)
• qBt-ratioLimit (string)
• qBt-previousSavePath (string)(maybe leave empty)
• qBt-seedDate (int) (secs from epoch)
• qBt-label (string)(only one)
• qBt-queuePosition (int)(-1 is used for completed torrents)
• qBt-seedStatus (int) (seed or not)

Also maybe share you ruby script? Although I am not familiar with Ruby I could catch an obvious error.

[briantist]

I was using 'runtime' in uTorrent to populate 'active_time'; I thought that fit best for a "Time active" field. I am definitely writing all times as BEncoded integers.

I am calculating the number of pieces as the size of the 'info'->'pieces' field in the .torrent divided by 20 (each SHA1 is 20 bytes). The math seemed to work out.

Then to populate 'piece_priority' I am just using 0x1 * npieces (it's what always seems to be set in the completed torrent fastresumes I looked at).

Populating pieces was a bit more difficult to figure out, but ended up being a one-liner in Ruby. From what I can tell, uT stores it in the 'have' field in resume.dat as a binary string, where each bit represents a piece. Of course that means it's padded up the nearest byte. qB uses the 'pieces' field for this and it's a binary string where each BYTE represents a piece, but the value is still just 0 or 1. Since we're using whole bytes though, it will always be npieces bytes long. So I just convert each bit to a byte, essentially: 0xFF becomes 0x0101010101010101

If any of that is incorrect, please let me know!

Thanks for the tip about 'paused'. I was setting 'auto_managed' to 1 all the time. What does that setting actually do? Should I just always make it 0, or should it just be the opposite of 'paused'? I changed the code to be opposite of 'paused' for now but don't have time to test it tonight.

The 3.1 changes sound really promising! With the addition of the save path and label it probably has everything I would need to do the import in one shot. Unfortunately I can't upgrade to 3.1 yet (political reasons, could be a few weeks). If you can send me a version of 3.1 that reports as 3.0.x to trackers I can test it sooner.

The ruby script as a while is a giant mess, because I started this with the assumption that I was going to use the web interface, maybe use watch folders, etc., so I've got loads of code that are testing one-off things with hard coded paths and names, lots of commented out crap. I can post the function that generates the .fastresume, that might help.

Let me look it over and I might post it tomorrow.

[sledgehammer_999]

From the libtorrent source (torrent.cpp)

[php]
// write have bitmask
// the pieces string has one byte per piece. Each
// byte is a bitmask representing different properties
// for the piece
// bit 0: set if we have the piece
// bit 1: set if we have verified the piece (in seed mode)
entry::string_type& pieces = ret["pieces"].string();
pieces.resize(m_torrent_file->num_pieces());
if (is_seed())
{
std::memset(&pieces[0], 1, pieces.size());
}
else
{
for (int i = 0, end(pieces.size()); i < end; ++i)
pieces = m_picker->have_piece(i) ? 1 : 0;
}

if (m_seed_mode)
{
TORRENT_ASSERT(m_verified.size() == pieces.size());
for (int i = 0, end(pieces.size()); i < end; ++i)
pieces |= m_verified ? 2 : 0;
}
[/php]

[briantist]

So essentially, 0xFF in uT 'have' should become 0x0303030303030303? (bits 0 and 1 set)?

[sledgehammer_999]

[quote="briantist"]
So essentially, 0xFF in uT 'have' should become 0x0303030303030303? (bits 0 and 1 set)?
[/quote]

Online calculators show that 11000000 (base2) is 0xC0(base16) or 192 (base10).

However, the comments and the source code don't match(I think). The above is correct if you follow the comment. If you follow the code this is correct:

00000011(base2) or 0x3(base16) or 3(base10)

In any case setting every bit to 1 should work, because libtorrent doesn't check the remaining 6bits in every byte/piece. Currently those bits don't represent something.