(Google's fault) Cryptomator missing most files and folders after Google Drive migration

I have the latest versions of Cryptomator (1.6.5) and macFUSE (4.2.4) installed on two Mac machines, and they both work well with my Dropbox and Google Drive accounts.

I recently migrated data from one Google account to another because the former account would expire soon. I used Google Takeout to create a copy of files on the account, which I could later transfer to the new account. After completing the transfer and ensuring all folders have the same size, I updated the paths to my Cryptomator vaults in the application. Unfortunately, when I unlock each vault on either machine, I only see a subset of files and folders in the vault.

What could be the cause of the problem, and how can it be resolved?

Thank you.

UPDATE 1: I just opened the problematic vaults on my iPhone and realized the files are not shown in my iPhone either (v2.0).

And your original Cryptomator vault is fully functional on the old account? Do you still have access to it? Do you have access to both vaults simultaneously?

Even though the size is the same, I suppose the directory structure is different. You can compare two directories: https://apple.stackexchange.com/a/228548

One example:

diff -rq vault-original/ vault-migrated/

If that’s not an option for you, you could also use the Vault Health Check tool in Cryptomator. It’s accessible under “Vault Options”.

Yes, the original vault is functioning properly.
The Vault Health Check tool does not show any errors when executed on the new vault.

I logged in to both accounts on Google Drive and ran a diff to see what has changed. The directory structure seems the same, but the file names are different. It looks like one or more characters from the end of the file names are trimmed, e.g.
0529dUA9SzcTNoa7ZvKAiGapuZjzzm3jYhh7yw8DmLB4v8U.c9r -------------------> new vault
0529dUA9SzcTNoa7ZvKAiGapuZjzzm3jYhh7yw8DmLB4v8UAeza38AE=.c9r —> original vault
Sometimes the missing character is just the = sign at the end of the file name (before the .c9r extension).

As you might imagine, changing the filenames is a huge problem. In that case, Cryptomator is unable to decrypt the filename. I guess Google Takeout doesn’t take the filenames too seriously and just trims however it wants. Is there any other way for you to move the vault?

1 Like

Yeah, I suppose the problem is on Google’s end.

I assume writing a script that renames files in the new vault would solve the problem, right?. All I need to do is to check the first few characters and file sizes, and hope there is a one-to-one mapping.

An alternative solution, which would take a long time for huge folders, is to copy files from one vault to another, essentially decrypting them from the original vault and re-encrypting them in the new vault.

That’s right. Maybe it’s even possible to generate a script out of the diff.

That would be possible as well, yes.

1 Like

Problem:
Google Takeout and Google Transfer make copies of files for account migration. However, during this process, they may trim some file names without giving any prior notice to the user. As a result, Cryptomator is unable to decrypt the file names and does not show most files when the vault is unlocked.
An example of such a change in a file name is seen below:
0529dUA9SzcTNoa7ZvKAiGapuZjzzm3jYhh7yw8DmLB4v8U.c9r -------------------> new vault
0529dUA9SzcTNoa7ZvKAiGapuZjzzm3jYhh7yw8DmLB4v8UAeza38AE=.c9r —> original vault

Solution

  1. Skip Google migration tools and migrate your files manually:
    1.1. Unlock the old and new vaults.
    1.2. Copy files from the old vault to the new vault.

This solution requires decrypting the contents of the first vault and re-encrypting them before writing them into the second vault. It also requires having enough space for both the original and new vaults.

  1. Let Google Transfer take care of migration and correct file names after the migration is done.
    2.1. After the transfer is done, log in to both accounts on Google Drive (desktop) and set the sync option to Stream mode. This prevents files from being downloaded to your machine.
    2.2. Run this Python script which will fix the names of files in the new vault.

I wrote this script without calling diff to ensure it is scalable for large vaults. Because some files may have correct names in the new vault, diff has to compare them, which slows the process down if those files are large. I generally found diff to be very slow for this purpose.
Instead, I developed a class based on Python’s dircmp and compared file names recursively to find mismatches. I expect this solution to be much faster than one based on diff.
I expect this script to work on different platforms.

If you find any issues in the code or have any suggestions about improving its performance or compatibility with different platforms, please let me know.

Thanks to @tobihagemann for helping me identify and resolve this issue.

2 Likes