File System Scan / Comparison Error

Hello,

I’m having an issue with Syncing around 200k files on a local drive to Cryptomator. I’m using a program called ViceVersa which scans the Source Drive, then scans the target drives, checks for differences, then deletes / copies files to the target. I am able to scan the Source fine, which is a local NTFS volume on my PC. When it attempts to scan the Target Drive, which is a Mounted Cryptomator Drive, stored on DropBox, it fails every time with the follow error: e:\Crypto*.* Insufficient system resources exist to complete the requested service.

I just built this machine yesterday. Its got 32gb of RAM, 10tb of physical storage free, an i5700k Gen9 Intel CPU, and an Nvidia 2070 Super. I’m pretty sure its not an actual resource issue, but I watched the resource monitor when I ran it the second time and nothing was maxed out even though it crashed at relatively the same point. We haven’t gotten to the file copying / deleting yet so there isn’t any decryption / encryption activity, at least according to the Cryptomator throughput monitor.

I contact the software developer for VeraCrypt and they told me to run dir /s at the command prompt which I did. It failed around the same point too, so they said it’s a problem with CryptoMator not their software. I am able to copy this same set of files to another PC without issues so I know its not my source.

I searched this forum and found that those who had a similar issue turned on the Custom Mount flags. I guess it had something to do with it timing out? I did that, and it took a lot longer for the error to come up, but it did.

Any ideas?

I just tried to delete the folder the Compare freezes at. Windows begins to count the files, then after 4k files, the windows disappears and volume is locked until I un-mount / remount it. The files never get deleted. Anyway to fix this or is it time to create a new volume that doesn’t have issues?

While waiting for the devs to provide assistance, I just wanted to say that I have been using Synchredible (Free) for all folder compare/sync jobs without issues (in case you need to try something different). I have not used it directly into an unlocked vault though, only for my backups between various disks.

But if both applications freeze at the same file/folder, then it is not an issue of the sync application you are using, at least you can exclude that possibility.

1 Like

Does switching between dokany and WebDAV have any influence on the behaviour?

Michael, thank you for the suggestion. I should have tried that. The comparison worked, although it took 45 minutes instead of 5. What does that mean? Should I continue to use WebDAV? Whats the Difference between that and Dokany? I prefer Dokany since it was so much, any ideas how I would get it to work?

I’m also get a lot of these errors now:

Copying “2019-12-25_05-39-05_20191225_053906.jpg” (6.42MB) … [Error renaming temp file W:\Z\Pictures\2019\Android~vv1.tmp. Unable to rename W:\Z\Pictures\2019\Android~vv1.tmp into W:\Z\Pictures\2019\Android\2019-12-25_05-39-05_20191225_053906.jpg. A device attached to the system is not functioning.] Error.
Copying “2019-12-25_05-39-10_20191225_053910.jpg” (6.32MB) …

But it doesn’t happen on every file.

Which dokany version do you have installed. If not latest, please uninstall it and update it and Switch settings back to dokany and check if this has an impact.

These are the virtual file systems that are used by Cryptomator to provide the virtual drives.
https://docs.cryptomator.org/en/latest/security/architecture/

Advantage of using dokany:

The error you are posting looks to me like there’s an issue either with the filesystem or with the hardware where the vault is located.

And of course I assume that you have the latest Cryptomator version installed?

I have the newest version of Dokany & Cryptomator. I’m just going to stick with WebDAV since it works. Thanks for your help!

Every Dokany mounted Vault has a timeout variable, giving a treshold in milliseconds. Everytime a call (for example “read file X”) to the virtual filesystem is made, a timer starts. It stops either if the call is handled (the file is actually read), or it hits the threshold, where the drive is unmounted forcefully. It’s a security measure of Dokany to not crash your machine when it starts choking on such calls.

As you already pointed out, the counter measures are to change the custom mount flags to

  1. increase the time out and/or
  2. increase the number of threads (these handle file system calls).

If there are a lot of calls to the file system, this may happen since even Dokany is faster than WebDAV, decryption takes its time. For example if you have 4 worker threads, and list currently 4 big directories, other calls cannot be handled and may cause a forceful unmount.

That is exactly what’s happening. I’ve tried to resolve it by using the flags and adding a 0 to the time out but its still unmounted forefully. Maybe that’s not enough? I’ll try to bump it even more and see if that does the trick.