Errors when trying to decrypt and copy all files out of my vault

Help Desktop App
1

I’d like to make a copy of the decrypted versions of the files in my vault, which is mounted as a WebDAV volume on my Mac (10.15.3). I’ve been trying to use rsync with the archive flag to copy them from the mounted volume, but I get various errors resulting in files that are skipped or incompletely transferred. Using the --progress flag with rsync I get errors such as:

rsync: read errors mapping "/Volumes/Work/_Math Tutoring/_BLOG/raw/IMG_5377.JPG": No message available on STREAM (96)

and also

WARNING: _Math Tutoring/Subjects - Algebra/IMG_5326.JPG failed verification -- update discarded (will try again).

I haven’t had this issue when copying files from other sources, so I wonder if it might be an issue with the vault. Any suggestions?

2

It might be an issue with the way rsync reads the files to estimate the file length. What switches are you using?

I am successfully using (a combination of) these switches:
-vrlptD --copy-dirlinks --exclude={…} ----filter=-rsp_{…}
without problems. I do not use --progress.
The -K, --keep-dirlinks switch does not work.

3

I’m just using rsync -av. The -a switch is supposed to be shorthand for -rlptgoD.

4

I’d recommend first trying to copy or open a failing file (such as the one quoted) in the vault. Not with rsync, just with a normal file manager/picture viewer. Might help to see if the issue is with the file in the vault, or if rsync is failing for some other reason.

1 Like
5

Correct. Though -rlptD omits g(roup) and o(ther) permissions, so it is not the same.

You said you use --progress. Does the problem only happen when using --progress?

6

It looks like good old tar was able to help me quite a bit. I used tar from the command line to create a massive archive file from the mounted WebDAV volume, and then unpacked this into my destination.

To check the results, I used:

diff -rq -x '.*' /Volumes/Cryptomator /Users/alexw/Documents/Cryptomator

There were still some files that didn’t make it into the archive, so I was able to copy most of the remaining ones using rsync without getting any errors:

rsync -av --exclude='.*' --progress --ignore-existing /Volumes/Cryptomator /Users/alexw/Documents

Note that for my purposes, I’ve excluded hidden . files.

I then ran diff one more time, and it seems that just about everything is synced now. A few caveats:

  • It looks like WebDAV is case sensitive while my Mac native filesystem is not. So I had a few files in the mounted volume that could not be copied because they had the same name but with different capitalization.
  • diff seems to sometimes have trouble detecting files containing extended unicode characters, so those may need to be checked with a different method.
7

I believe it was having trouble with or without the flag. It seemed like one of the issues was in copying larger raw video files, although those weren’t the only types of files that generated errors.