Page 1 of 1

Script: Bulk Change Save Path

Posted: Sat Apr 01, 2017 3:22 am
by jslay
This has been packaged up and maintained now through PyPi.

Install Python and follow the README at https://github.com/jslay88/qbt_migrate

TL;DR

Once Python is installed, open a terminal/command prompt and run the following

Code: Select all

pip install qbt_migrate
qbt_migrate
Follow on-screen prompts.

Re: Script: Bulk Change Save Path

Posted: Thu Apr 27, 2017 2:49 pm
by jonboy345
Just wanted to thank you for this...

Saved me a TON of time getting my data moved to a new NAS.

Thank you, thank you, thank you!

Re: Script: Bulk Change Save Path

Posted: Thu Jun 15, 2017 8:58 pm
by cfsenel
Thank you very much, it was exactly what I needed. (I just created this account to thank you.) Just a small correction: towards the end, the line

Code: Select all

rear_string[sp_pos_stop:]
should be

Code: Select all

rear_string = data[sp_pos_stop:]

Re: Script: Bulk Change Save Path

Posted: Wed Jan 03, 2018 12:40 am
by rysvald
Thank you very much for this!

I just want to add that it works perfectly with cfsenels correction and that it has no problem handling different path lengths like this:
X:\ --> X:\Torrents\
or
X:\Torrents\ --> X:\

Re: Script: Bulk Change Save Path

Posted: Fri Jan 12, 2018 8:25 am
by L0ki
This is awesome, thanks so much for developing this. It was a real lifesaver as I was having trouble just changing this using notepad++ I would have had to readd a thousand torrents were it not for this tool.

Re: Script: Bulk Change Save Path

Posted: Sun May 19, 2019 9:28 am
by jslay
Added a Python3 script since Python2 is EOL Jan 1 2020.

Python3 version will automatically backup the file before overwriting it.
Python3 version also will find the BT_backup folder for the current user.

Feel free to import/modify as you wish.

Re: Script: Bulk Change Save Path

Posted: Fri May 24, 2019 1:34 am
by heapson
Super huge thanks for the new script. When I first needed this, I had troubles using the original script because I installed Python 3. Took me a while to work that out.

This time I needed to move 3,000 torrents scattered across various local drives to a central NAS and it saved me a ton of time. Cheers!

With the V2 script, I encountered the error, 'NoneType' object has no attribute 'start' and the script halted. I still had hundreds of torrents to sort out for one specific root location but the sub folders meant qBt couldn't handle it en-mass.

I switched to V3 but encountered similar errors, ValueError('Unable to determine \'save_path\'')

I added "print(self.file_path)" to the script and discovered it was a single fastresume file causing the glitch. After deleting the offending file, the script completed perfectly. Happy days!

I thought I'd add my experience in case anyone else encounters the issue with a corrupt fastresume file.

Edit: After comparing the file with a good one, it's clearly truncated just before the save_path field so it makes perfect sense the script failed.

Code: Select all

        qbt_matches = self._pattern_qBt_save_path.search(self._data)
        print(self.file_path)
        if save_path_matches is None:
            raise ValueError('Unable to determine \'save_path\'')

Re: Script: Bulk Change Save Path

Posted: Sun Nov 03, 2019 9:01 am
by Wepeel
Is there any way to make this work with non-ASCII characters in the path?

I have paths that kind of looks like this D:\ÅÅÅ\ÄÄÄ\ÖÖÖ\ and if I want to do a drive letter change the torrent breaks and doesn't show up in qBittorrent anymore. It works otherwise, but if one sub-folder has one of these characters it doesn't seem to work.

Re: Script: Bulk Change Save Path

Posted: Sun Nov 03, 2019 5:59 pm
by Peter
Wepeel wrote: Sun Nov 03, 2019 9:01 am...
You ran the Python3 version?

Re: Script: Bulk Change Save Path

Posted: Mon Nov 04, 2019 5:42 am
by Wepeel
Peter wrote: Sun Nov 03, 2019 5:59 pm
You ran the Python3 version?
Yeah, I used the Python3 version.

Re: Script: Bulk Change Save Path

Posted: Tue Nov 05, 2019 5:54 am
by jairinhohw
I fixed the problem with non-ascii characters.
The problem was an error in the calculation of the length of the new path, because it was calculating the length of the string instead of the utf-8 bytes.

Code: Select all

import os
import re
import sys
import logging
import zipfile

from datetime import datetime


logger = logging.getLogger(__name__)


class QBTBatchMove(object):
    def __init__(self, bt_backup_path=None):
        if bt_backup_path is None:
            logger.debug('Discovering BT_backup path...')
            if sys.platform.startswith('win32'):
                logger.debug('Windows System')
                bt_backup_path = os.path.join(
                    os.getenv('localappdata'), 'qBittorrent\\BT_backup')
            elif sys.platform.startswith('linux'):
                logger.debug('Linux System')
                bt_backup_path = os.path.join(
                    os.getenv('HOME'), '.local/share/data/qBittorrent/BT_backup')
        if not os.path.exists(bt_backup_path) or not os.path.isdir(bt_backup_path):
            raise NotADirectoryError(bt_backup_path)
        logger.debug('BT_backup Path: %s' % bt_backup_path)
        self.bt_backup_path = bt_backup_path
        self.discovered_files = None

    def run(self, existing_path, new_path, create_backup=True):
        """
        Perform Batch Processing of path changes.
        :param existing_path: Existing path to look for
        :type existing_path: str
        :param new_path: New Path to replace with
        :type new_path: str
        :param create_backup: Create a backup of the file before modifying?
        :type create_backup: bool
        """
        if create_backup:
            backup_filename = 'fastresume_backup' + \
                datetime.now().strftime('%Y%m%d%H%M%S') + '.zip'
            self.backup_folder(self.bt_backup_path,
                               os.path.join(os.path.dirname(self.bt_backup_path), backup_filename))
        if sys.platform.startswith('win32') and not existing_path.endswith('\\'):
            existing_path += '\\'
        elif sys.platform.startswith('linux') and not existing_path.endswith('/'):
            existing_path += '/'
        if sys.platform.startswith('win32') and not new_path.endswith('\\'):
            new_path += '\\'
        elif sys.platform.startswith('linux') and not new_path.endswith('/'):
            new_path += '/'
        logger.info('Searching for .fastresume files with path %s ...' %
                    existing_path)
        self.discovered_files = self.discover_relevant_fast_resume(self.bt_backup_path,
                                                                   existing_path)
        if not self.discovered_files:

            raise ValueError(
                'Found no .fastresume files with existing path %s' % existing_path)

        logger.info('Found %s Files.' % len(self.discovered_files))
        logger.debug('Discovered FastResumes: %s' % self.discovered_files)

        for file in self.discovered_files:

            logger.info('Updating File %s...' % file.file_path)
            new_save_path = file.save_path.replace(bytes(existing_path, 'utf-8'),
                                                   bytes(new_path, 'utf-8'))

            file.set_save_path(new_save_path.decode('utf-8'),
                               save_file=False,
                               create_backup=False)

            new_qbt_save_path = file.qbt_save_path.replace(bytes(existing_path, 'utf-8'),
                                                           bytes(new_path, 'utf-8'))

            file.set_qbt_save_path(new_qbt_save_path.decode('utf-8'),
                                   save_file=False,
                                   create_backup=False)

            file.save()
            logger.info('File (%s) Updated!' % file.file_path)

    @staticmethod
    def discover_relevant_fast_resume(bt_backup_path, existing_path):
        """
        Find .fastresume files that contain the existing path.
        :param bt_backup_path: Path to BT_backup folder
        :type bt_backup_path: str
        :param existing_path: The existing path to look for
        :type existing_path: str
        :return: List of FastResume Objects
        :rtype: list[FastResume]
        """
        existing_path = bytes(existing_path, 'utf-8')
        fast_resume_files = [FastResume(os.path.join(bt_backup_path, file))
                             for file in os.listdir(bt_backup_path)
                             if file.endswith('.fastresume')]
        fast_resume_files = [file for file in fast_resume_files if existing_path in file.save_path
                             or existing_path in file.qbt_save_path]
        return fast_resume_files

    @staticmethod
    def backup_folder(folder_path, archive_path):
        logger.info('Creating Archive %s ...' % archive_path)
        with zipfile.ZipFile(archive_path, 'w') as archive:
            for file in os.listdir(folder_path):
                archive.write(os.path.join(folder_path, file))
        logger.info('Done!')


class FastResume(object):
    save_path_pattern = br'save_path(\d+)'
    qBt_save_path_pattern = br'qBt-savePath(\d+)'

    def __init__(self, file_path):
        self.file_path = os.path.realpath(file_path)
        if not os.path.exists(self.file_path) or not os.path.isfile(self.file_path):
            raise FileNotFoundError(self.file_path)
        self._save_path = None
        self._qbt_save_path = None
        self._pattern_save_path = re.compile(self.save_path_pattern)
        self._pattern_qBt_save_path = re.compile(self.qBt_save_path_pattern)
        self._data = None
        self._load_data()

    def _load_data(self):
        with open(self.file_path, 'rb') as f:
            self._data = f.read()
        save_path_matches = self._pattern_save_path.search(self._data)
        qbt_matches = self._pattern_qBt_save_path.search(self._data)
        if save_path_matches is None:
            raise ValueError('Unable to determine \'save_path\'')
        if qbt_matches is None:
            raise ValueError('Unable to determine \'qBt-savePath\'')
        self._save_path = self._data[save_path_matches.end() + 1:
                                     save_path_matches.end() + 1 + int(save_path_matches.group(1))]
        self._qbt_save_path = self._data[qbt_matches.end() + 1:
                                         qbt_matches.end() + 1 + int(qbt_matches.group(1))]

    @property
    def save_path(self):
        return self._save_path

    @property
    def qbt_save_path(self):
        return self._qbt_save_path

    def set_save_path(self, path, save_file=True, create_backup=True):
        if sys.platform.startswith('win32') and not path.endswith('\\'):
            path += '\\'
        elif sys.platform.startswith('linux') and not path.endswith('/'):
            path += '/'
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self._data = self._data.replace(
            b'save_path' + bytes(str(len(self._save_path)),
                                 'utf-8') + b':' + self._save_path,
            b'save_path' + bytes(str(len(bytes(path, 'utf-8'))), 'utf-8') +
            b':' + bytes(path, 'utf-8')
        )
        self._save_path = bytes(path, 'utf-8')
        if save_file:
            self.save()

    def set_qbt_save_path(self, path, save_file=True, create_backup=True):
        if sys.platform.startswith('win32') and not path.endswith('\\'):
            path += '\\'
        elif sys.platform.startswith('linux') and not path.endswith('/'):
            path += '/'
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self._data = self._data.replace(
            b'qBt-savePath' +
            bytes(str(len(self._qbt_save_path)), 'utf-8') +
            b':' + self._qbt_save_path,
            b'qBt-savePath' + bytes(str(len(bytes(path, 'utf-8'))), 'utf-8') +
            b':' + bytes(path, 'utf-8')
        )
        self._qbt_save_path = bytes(path, 'utf-8')
        if save_file:
            self.save()

    def set_save_paths(self, path, save_file=True, create_backup=True):
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self.set_save_path(path, save_file=False, create_backup=False)
        self.set_qbt_save_path(path, save_file=False, create_backup=False)
        if save_file:
            self.save()

    def save(self, file_name=None):
        if file_name is None:
            file_name = self.file_path
        logger.info('Saving File %s...' % file_name)
        with open(file_name, 'wb') as f:
            f.write(self._data)


if __name__ == '__main__':
    logger.addHandler(logging.StreamHandler(stream=sys.stdout))
    logger.setLevel('INFO')
    qbm = QBTBatchMove()
    bt_backup = input('BT_Backup Path (%s): ' % qbm.bt_backup_path)
    if bt_backup != '':
        qbm.bt_backup_path = bt_backup

    if len(sys.argv) > 1:
        ep = sys.argv[1]
        print('Existing Path: (%s)' % ep)
    else:
        ep = input('Existing Path: ')

    if len(sys.argv) > 2:
        np = sys.argv[2]
        print('New Path: (%s)' % np)
    else:
        np = input('New Path: ')

    qbm.run(ep, np)


Re: Script: Bulk Change Save Path

Posted: Tue Nov 05, 2019 5:57 am
by jairinhohw
Wepeel wrote: Sun Nov 03, 2019 9:01 am Is there any way to make this work with non-ASCII characters in the path?

I have paths that kind of looks like this D:\ÅÅÅ\ÄÄÄ\ÖÖÖ\ and if I want to do a drive letter change the torrent breaks and doesn't show up in qBittorrent anymore. It works otherwise, but if one sub-folder has one of these characters it doesn't seem to work.
I fixed the problem with non-ascii characters.
The problem was an error in the calculation of the length of the new path, because it was calculating the length of the string instead of the utf-8 bytes.

Code: Select all

import os
import re
import sys
import logging
import zipfile

from datetime import datetime


logger = logging.getLogger(__name__)


class QBTBatchMove(object):
    def __init__(self, bt_backup_path=None):
        if bt_backup_path is None:
            logger.debug('Discovering BT_backup path...')
            if sys.platform.startswith('win32'):
                logger.debug('Windows System')
                bt_backup_path = os.path.join(
                    os.getenv('localappdata'), 'qBittorrent\\BT_backup')
            elif sys.platform.startswith('linux'):
                logger.debug('Linux System')
                bt_backup_path = os.path.join(
                    os.getenv('HOME'), '.local/share/data/qBittorrent/BT_backup')
        if not os.path.exists(bt_backup_path) or not os.path.isdir(bt_backup_path):
            raise NotADirectoryError(bt_backup_path)
        logger.debug('BT_backup Path: %s' % bt_backup_path)
        self.bt_backup_path = bt_backup_path
        self.discovered_files = None

    def run(self, existing_path, new_path, create_backup=True):
        """
        Perform Batch Processing of path changes.
        :param existing_path: Existing path to look for
        :type existing_path: str
        :param new_path: New Path to replace with
        :type new_path: str
        :param create_backup: Create a backup of the file before modifying?
        :type create_backup: bool
        """
        if create_backup:
            backup_filename = 'fastresume_backup' + \
                datetime.now().strftime('%Y%m%d%H%M%S') + '.zip'
            self.backup_folder(self.bt_backup_path,
                               os.path.join(os.path.dirname(self.bt_backup_path), backup_filename))
        if sys.platform.startswith('win32') and not existing_path.endswith('\\'):
            existing_path += '\\'
        elif sys.platform.startswith('linux') and not existing_path.endswith('/'):
            existing_path += '/'
        if sys.platform.startswith('win32') and not new_path.endswith('\\'):
            new_path += '\\'
        elif sys.platform.startswith('linux') and not new_path.endswith('/'):
            new_path += '/'
        logger.info('Searching for .fastresume files with path %s ...' %
                    existing_path)
        self.discovered_files = self.discover_relevant_fast_resume(self.bt_backup_path,
                                                                   existing_path)
        if not self.discovered_files:

            raise ValueError(
                'Found no .fastresume files with existing path %s' % existing_path)

        logger.info('Found %s Files.' % len(self.discovered_files))
        logger.debug('Discovered FastResumes: %s' % self.discovered_files)

        for file in self.discovered_files:

            logger.info('Updating File %s...' % file.file_path)
            new_save_path = file.save_path.replace(bytes(existing_path, 'utf-8'),
                                                   bytes(new_path, 'utf-8'))

            file.set_save_path(new_save_path.decode('utf-8'),
                               save_file=False,
                               create_backup=False)

            new_qbt_save_path = file.qbt_save_path.replace(bytes(existing_path, 'utf-8'),
                                                           bytes(new_path, 'utf-8'))

            file.set_qbt_save_path(new_qbt_save_path.decode('utf-8'),
                                   save_file=False,
                                   create_backup=False)

            file.save()
            logger.info('File (%s) Updated!' % file.file_path)

    @staticmethod
    def discover_relevant_fast_resume(bt_backup_path, existing_path):
        """
        Find .fastresume files that contain the existing path.
        :param bt_backup_path: Path to BT_backup folder
        :type bt_backup_path: str
        :param existing_path: The existing path to look for
        :type existing_path: str
        :return: List of FastResume Objects
        :rtype: list[FastResume]
        """
        existing_path = bytes(existing_path, 'utf-8')
        fast_resume_files = [FastResume(os.path.join(bt_backup_path, file))
                             for file in os.listdir(bt_backup_path)
                             if file.endswith('.fastresume')]
        fast_resume_files = [file for file in fast_resume_files if existing_path in file.save_path
                             or existing_path in file.qbt_save_path]
        return fast_resume_files

    @staticmethod
    def backup_folder(folder_path, archive_path):
        logger.info('Creating Archive %s ...' % archive_path)
        with zipfile.ZipFile(archive_path, 'w') as archive:
            for file in os.listdir(folder_path):
                archive.write(os.path.join(folder_path, file))
        logger.info('Done!')


class FastResume(object):
    save_path_pattern = br'save_path(\d+)'
    qBt_save_path_pattern = br'qBt-savePath(\d+)'

    def __init__(self, file_path):
        self.file_path = os.path.realpath(file_path)
        if not os.path.exists(self.file_path) or not os.path.isfile(self.file_path):
            raise FileNotFoundError(self.file_path)
        self._save_path = None
        self._qbt_save_path = None
        self._pattern_save_path = re.compile(self.save_path_pattern)
        self._pattern_qBt_save_path = re.compile(self.qBt_save_path_pattern)
        self._data = None
        self._load_data()

    def _load_data(self):
        with open(self.file_path, 'rb') as f:
            self._data = f.read()
        save_path_matches = self._pattern_save_path.search(self._data)
        qbt_matches = self._pattern_qBt_save_path.search(self._data)
        if save_path_matches is None:
            raise ValueError('Unable to determine \'save_path\'')
        if qbt_matches is None:
            raise ValueError('Unable to determine \'qBt-savePath\'')
        self._save_path = self._data[save_path_matches.end() + 1:
                                     save_path_matches.end() + 1 + int(save_path_matches.group(1))]
        self._qbt_save_path = self._data[qbt_matches.end() + 1:
                                         qbt_matches.end() + 1 + int(qbt_matches.group(1))]

    @property
    def save_path(self):
        return self._save_path

    @property
    def qbt_save_path(self):
        return self._qbt_save_path

    def set_save_path(self, path, save_file=True, create_backup=True):
        if sys.platform.startswith('win32') and not path.endswith('\\'):
            path += '\\'
        elif sys.platform.startswith('linux') and not path.endswith('/'):
            path += '/'
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self._data = self._data.replace(
            b'save_path' + bytes(str(len(self._save_path)),
                                 'utf-8') + b':' + self._save_path,
            b'save_path' + bytes(str(len(bytes(path, 'utf-8'))), 'utf-8') +
            b':' + bytes(path, 'utf-8')
        )
        self._save_path = bytes(path, 'utf-8')
        if save_file:
            self.save()

    def set_qbt_save_path(self, path, save_file=True, create_backup=True):
        if sys.platform.startswith('win32') and not path.endswith('\\'):
            path += '\\'
        elif sys.platform.startswith('linux') and not path.endswith('/'):
            path += '/'
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self._data = self._data.replace(
            b'qBt-savePath' +
            bytes(str(len(self._qbt_save_path)), 'utf-8') +
            b':' + self._qbt_save_path,
            b'qBt-savePath' + bytes(str(len(bytes(path, 'utf-8'))), 'utf-8') +
            b':' + bytes(path, 'utf-8')
        )
        self._qbt_save_path = bytes(path, 'utf-8')
        if save_file:
            self.save()

    def set_save_paths(self, path, save_file=True, create_backup=True):
        if create_backup:
            today = datetime.now()
            file_name = '%s.%s.%s' % (self.file_path,
                                      today.strftime('%Y%m%d%H%M%S'),
                                      'bkup')
            self.save(file_name)
        self.set_save_path(path, save_file=False, create_backup=False)
        self.set_qbt_save_path(path, save_file=False, create_backup=False)
        if save_file:
            self.save()

    def save(self, file_name=None):
        if file_name is None:
            file_name = self.file_path
        logger.info('Saving File %s...' % file_name)
        with open(file_name, 'wb') as f:
            f.write(self._data)


if __name__ == '__main__':
    logger.addHandler(logging.StreamHandler(stream=sys.stdout))
    logger.setLevel('INFO')
    qbm = QBTBatchMove()
    bt_backup = input('BT_Backup Path (%s): ' % qbm.bt_backup_path)
    if bt_backup != '':
        qbm.bt_backup_path = bt_backup

    if len(sys.argv) > 1:
        ep = sys.argv[1]
        print('Existing Path: (%s)' % ep)
    else:
        ep = input('Existing Path: ')

    if len(sys.argv) > 2:
        np = sys.argv[2]
        print('New Path: (%s)' % np)
    else:
        np = input('New Path: ')

    qbm.run(ep, np)

(Sorry if it was sent more than one time, i think I've sent before, but couldn't find the reply)

Re: Script: Bulk Change Save Path

Posted: Tue Nov 19, 2019 11:32 pm
by stockley
I'm trying to automate a move from Windows to Linux.

As far as I can see, this script only changes the "save_path", but it leaves the "mapped_files" path unchanged?

Different nested files and/or folders will therefor be invalid because they will still have the Windows backslashes.

So I am testing another script (based on https://github.com/ctminime/QB_Migrate_to_Linux ) so the "mapped_files" within a torrent will also be changed.

Unfortunately, I am ending up with incorrect data in the re-encoded fastresume file. I think the problem lies here:

Code: Select all

mapped_files_modified = re.sub("\\\\", "/",str(x))
Anyone care to take a look?

full script:

Code: Select all

import os
import bencode
import re
#---Data directory of QBittorrent fastresume files
tor_dir = "/home/wim/.local/share/data/qBittorrent/BT_backup/"
#---Work on files with  extension .fastresume
for file in os.listdir(tor_dir):
 if file.endswith(".fastresume"):
#---File name per file
  file_path_name = os.path.join(tor_dir, file)
#---Load contents of file
  torrent = bencode.bread(file_path_name)
#---Define x as the 'mapped_files' field
  x = torrent.get('mapped_files')
#---Uncomment for testing
#  print(x)
#---replace Windows blackslash with Linux fw slash
  mapped_files_modified = re.sub("\\\\", "/",str(x))
#---Uncomment for testing
#  print(mapped_files_modified)
#---Replace paths in loaded decoded data
  torrent['mapped_files']=mapped_files_modified
#---Uncomment for testing
#  print(torrent['mapped_files'])
#---Write modified file
  bencode.bwrite(torrent, file_path_name)

Re: Script: Bulk Change Save Path

Posted: Sat Feb 29, 2020 8:12 am
by jslay
I have updated the script to support setting the target OS, which will subsequently convert the slashes.

https://github.com/jslay88/qbt_migrate

Re: Script: Bulk Change Save Path

Posted: Sun Jan 14, 2024 4:16 pm
by versita
Thank you. This script has saved me from having to recheck 10+ terabytes. I was migrating my qB client between hosts and the torrents wouldn't fast resume because of the change in save paths.