I/O Errors and file differences

First of all: Thank you for Version 1.5.0 and 1.5.1! I really like the new User Interface and the new version allows me to open more than 3 or 4 vaults at the same time.

OS: macOS Catalina 10.15.5 Beta (19F62f)
Cryptomator: Version 1.5.1
FUSE: 3.10.4
Hardware: The experience reported here is from a MacBook Pro and an iMac (64GB RAM)

Cryptomator usage: Currently I have 7 Cryptomator vaults. 4 vaults are on USB memory sticks of various sizes (16GB, 128GB, 256GB), speeds and interface types (USB, USB 3.0, USB-C). The USB sticks are GUID partitioned, the file system is APFS (encrypted).

  • 1 Cryptomator vault is on Microsoft OneDrive {Version 19.232.1124.0012 (eigenständig)}
  • 1 Cryptomator vault is on Google Drive (current Version)
  • 1 Cryptomator vault is on iCloud Drive

The Microsoft OneDrive vault is the one I work with (Master). Here I change/add/delete files. All other vaults are just copies of the Master. In addition I keep one unencrypted copy of the Master in the local filesystem. From time to time I update the copies of the Master. The method I use is to rsync the unencrypted content between the vaults.
rsync --archive --hard-links --whole-file --one-file-system --verbose --delete --checksum <source> <destination>.
The unencrypted vaults contain 13000 files and 1200 directories, occupying 8GB of disk space.
Before syncing the content, I check for differences. I use two methods: rsync with the same options as above plus --dry-run and diff --recursive <source> <destination>. I do this to check if what rsync would change is plausible. In my opinion it is absolutely necessary to do these checks because I experienced some strange behaviour regarding the vaults.
Sometimes rsync --dry-run or diff --recursive report an I/O error or differences between the source and destination for a file that has not been touched for a couple of years. When I check the file that reported an I/O error or a difference manually, the file is readable, has the expected content and in case of difference there is none.
There was one exception when a difference was reported between source and destination which was persistent. The file in the OneDrive vault (Master) had not been touched but had the size and content of a different file in the same directory. After I closed the OneDrive based vault and reopened it, the file had the correct size and content.
In summary it appears that sometimes files in a Cryptomator vault produce I/O errors or wrong content is read from them in a non repeatable way. On the memory sticks I sometimes got ‘device not ready’ errors which were also not repeatable. With not repeatable I mean I found size and content of reported files to be o.k. immediately after the error message showed up.

In order to investigate the behavior further, I automated testing differences between the vaults and let run for more than 12h:

crryptomatorsync.bash is a script I use to sync and check for differences. First parameter is the name of a vault or directory.

  • OneDriveCopy is an unencrypted copy of the vault content in the file system (APFS encrypted)
  • iCloudDrive is a Cryptomator vault in the iCloud directory
  • SanDisk256GB is a Cryptomator vault on a memory stick

It can be seen in the below run there is an I/O error in the Master, also differences between the Master and other vaults. Without any action on my side, no I/O error and no differences are reported on the following runs.
The next I/O error is reported in the iCloud vault at 07:39 the next day. Shortly before, I had made some minor changes (5 files) to the Master and synced them with the other vaults. No error in the iCloud vault is reported on runs after that one. But at 08:59 a file difference is reported between the Master and Google Drive. The file has not been touched in years and no difference is reported in the runs after. (The last two incidents are not in the output below.)

I have the impression (gut feeling) immediately after opening (or perhaps first use after opening) a vault these errors occur more often. Also there seems to be a cumulation of those errors when the file system is busy while I made Carbon Copy Cloner disk backups of the system disk the OneDrive, Google Drive and iCloud Drive vaults reside on.

> while true;
>     do for f in OneDriveCopy iCloudDrive SanDisk256GB ;
>         do echo -e "########################################\n###\n### $f\n###";
>             date;
>             time cryptomatorsync.bash $f difference;
>         done ;
>     done;
> ########################################
> ###
> ### OneDriveCopy
> ###
> Sa 25 Apr 2020 20:05:54 CEST
> diff: /Users/lange/Cryptomator/OneDrive/Oracle/PerformanceTuning/\<removed for privacy\>/sqlpath/where1.sql: Input/output error
> Files /Users/lange/Cryptomator/OneDrive/Oracle/Projects/HAWorkshop/Oracle12cRAC_Installation.key/Data/12cRACInstallation243-small.png and /Volumes/TimeMachine/Media/OneDriveCopy/Oracle/Projects/HAWorkshop/Oracle12cRAC_Installation.key/Data/12cRACInstallation243-small.png differ
> 
> real	1m55.745s
> user	0m0.907s
> sys	0m12.920s
> ########################################
> ###
> ### iCloudDrive
> ###
> Sa 25 Apr 2020 20:07:49 CEST
> diff: /Users/lange/Cryptomator/iCloudDrive/Oracle/Projects/HAWorkshop/Oracle12cRAC_Installation.key/Data/12cRACInstallation183-small.png: Input/output error
> diff: /Users/lange/Cryptomator/iCloudDrive/Oracle/knowhow/Antognini/Scripts20100624/databases/DBM11106/DBM11106.sql: Input/output error
> 
> real	1m52.768s
> user	0m0.931s
> sys	0m14.509s
> ########################################
> ###
> ### SanDisk256GB
> ###
> Sa 25 Apr 2020 20:09:42 CEST
> 
> real	2m43.075s
> user	0m0.929s
> sys	0m14.949s
> ########################################
> ###
> ### OneDriveCopy
> ###
> Sa 25 Apr 2020 20:12:25 CEST
> 
> real	1m12.700s
> user	0m0.770s
> sys	0m3.402s
> ########################################
> ###
> ### iCloudDrive
> ###
> Sa 25 Apr 2020 20:13:38 CEST
> 
> real	0m23.925s
> user	0m0.756s
> sys	0m3.729s
> ########################################
> ###
> ### SanDisk256GB
> ###
> Sa 25 Apr 2020 20:14:02 CEST
> 
> real	0m45.334s
> user	0m0.807s
> sys	0m4.321s

Regarding the differences: You might find this interesting: