Vorta cannot recover from broken cache

[quazgar]

Description

Somehow my machine crashed during backup, with the result that there were files with invalid content in ~/.var/app/com.borgbase.Vorta/cache/borg/12345.... When attempting the next backup, Vorta seemed to not catch borg's exception and leave behind a lock directory. This lock then in the next backup attempt at least notifies the user, but of course even Vorta's borg break-lock does not resolve the issue of the files with invalid content.

Reproduce

1. Create new minimal backup profile which backs up some data to some local directory (some folder in `/tmp/ is sufficent).
2. Run a backup.
3. Modify the content of ~/.var/app/com.borgbase.Vorta/cache/borg/12345678abcdef/chunks (use the correct hash)
4. Attempt another backup run. This will fail with the exception as seen in the logs.

Workaround

What helped was to purge the local cache:

# --remote-path /usr/local/bin/borg was necessary in my case, but YMMV
BORG_CACHE_DIR=~/.var/app/com.borgbase.Vorta/cache/borg/ borg delete --remote-path /usr/local/bin/borg --cache-only ssh://borg-backup@mybackupserver/path/to borg/repo

Reproduction

• I tried to reproduce the issue.
• I was able to reproduce the issue.

OS

Linux (Debian 12)

Version of Vorta

0.9.1

What did you install Vorta with?

Flatpak

Version of Borg

1.2.4

Logs

2024-12-02 22:53:47,644 - vorta.scheduler - DEBUG - Refreshing all scheduler timers
2024-12-02 22:53:47,652 - vorta.scheduler - INFO - Setting timer for profile 1
2024-12-02 22:53:47,653 - vorta.scheduler - DEBUG - Catching up by running job for home-mybackup-select (1)
2024-12-02 22:53:47,663 - vorta.scheduler - INFO - Starting background backup for home-mybackup-select
2024-12-02 22:53:47,664 - vorta.notifications - DEBUG - success notifications suppressed
2024-12-02 22:53:47,666 - vorta.keyring.abc - DEBUG - No module named 'objc'
2024-12-02 22:53:47,691 - vorta.keyring.abc - DEBUG - Using VortaKWallet5Keyring
2024-12-02 22:53:47,692 - vorta.borg.borg_job - DEBUG - Using VortaKWallet5Keyring keyring to store passwords.
2024-12-02 22:53:47,697 - vorta.keyring.kwallet - DEBUG - Retrieved password for repo ssh://borg-backup@mybackupserver/volume1/borgbackup/myrepo
2024-12-02 22:53:47,723 - vorta.scheduler - INFO - Preparation for backup successful.
2024-12-02 22:53:47,724 - vorta.borg.jobs_manager - DEBUG - Add job for site 2
2024-12-02 22:53:47,725 - vorta.borg.jobs_manager - DEBUG - Start job on site: 2
2024-12-02 22:53:47,800 - vorta.borg.borg_job - INFO - Running command /app/bin/borg create --remote-path=/usr/local/bin/borg --list --progress --info --log-json --json --filter=AM -C lz4 --exclude-from /tmp/tmp75iazl2l ssh://borg-backup@mybackupserver/volume1/borgbackup/myrepo::myhost-2024-12-02-225347 /home/myuser/Pictures /home/myuser/Documents
2024-12-02 22:54:18,801 - vorta.borg.borg_job - INFO - Creating archive at "ssh://borg-backup@mybackupserver/volume1/borgbackup/myrepo::myhost-2024-12-02-225347"
2024-12-02 22:54:19,070 - vorta.borg.borg_job - ERROR - File failed integrity check: /home/myuser/.var/app/com.borgbase.Vorta/cache/borg/123456789abcdef/chunks
2024-12-02 22:54:19,071 - vorta.borg.borg_job - ERROR - Traceback (most recent call last):
  File "/app/lib/python3.11/site-packages/borg/archiver.py", line 5213, in main
    exit_code = archiver.run(args)
                ^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/archiver.py", line 5144, in run
    return set_ec(func(args))
                  ^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/archiver.py", line 656, in do_create
    with Cache(repository, key, manifest, progress=args.progress,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 387, in __new__
    return local()
           ^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 378, in local
    return LocalCache(repository=repository, key=key, manifest=manifest, path=path, sync=sync,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 487, in __init__
    self.open()
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 542, in open
    self.rollback()
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 662, in rollback
    self._do_open()
  File "/app/lib/python3.11/site-packages/borg/cache.py", line 527, in _do_open
    self.chunks = ChunkIndex.read(fd)
                  ^^^^^^^^^^^^^^^^^^^
  File "src/borg/hashindex.pyx", line 114, in borg.hashindex.IndexBase.read
  File "src/borg/hashindex.pyx", line 99, in borg.hashindex.IndexBase.__cinit__
  File "/app/lib/python3.11/site-packages/borg/crypto/file_integrity.py", line 189, in hash_part
    raise FileIntegrityError(self.path)
borg.crypto.file_integrity.FileIntegrityError: File failed integrity check: /home/myuser/.var/app/com.borgbase.Vorta/cache/borg/123456789abcdef/chunks

Platform: Linux myhost 6.1.0-26-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.112-1 (2024-09-30) x86_64
Linux: Unknown Linux  
Borg: 1.2.4  Python: CPython 3.11.7 msgpack: 1.0.4 fuse: pyfuse3 3.2.2 [pyfuse3,llfuse]
PID: 34  CWD: /home/myuser
sys.argv: ['/app/bin/borg', 'create', '--remote-path=/usr/local/bin/borg', '--list', '--progress', '--info', '--log-json', '--json', '--filter=AM', '-C', 'lz4', '--exclude-from', '/tmp/tmp75iazl2l', 'ssh://borg-backup@mybackupserver/volume1/borgbackup/myrepo::myhost-2024-12-02-225347', '/home/myuser/Pictures', '/home/myuser/Documents']
SSH_ORIGINAL_COMMAND: None

2024-12-02 22:54:19,208 - vorta.borg.jobs_manager - DEBUG - Finish job for site: 2
2024-12-02 22:54:19,208 - vorta.borg.jobs_manager - DEBUG - No more jobs for site: 2