Slow encryption of large amount of files

I was trying to upload a folder with several java projects to a cloud storage, totally 3408 files. At around 70% the process of copying practically stopped and Cryptomator app seemed to become frozen.
Π‘Π½ΠΈΠΌΠΎΠΊ экрана 2021-12-12 000248

So is there any amount of files limit? Or the problem is connected with something else?

No there isn’t. The problem must be something else.
Which virtual drive type are you using? WebDAV? Dokany? WinSFP?
Does switching the volume type change anything?
Anything suspicious in the log file when the process is stopping?

I have only WebDAV option is settings, so I can’t switch the volume type. And yeah there’s this thing in the log

16:00:08.375 [webdav-024] WARN  org.eclipse.jetty.server.HttpChannel - /XOA92yDuiX3Z/bbb/IdeaProjects/projectone/.git/objects/c0/8b0d14221955e5d6a886e8fbcb57976f26e95f
com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2053)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache.get(LocalCache.java:3966)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4863)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkCache.get(ChunkCache.java:60)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.CleartextFileChannel.writeLockedInternal(CleartextFileChannel.java:159)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.CleartextFileChannel.writeLocked(CleartextFileChannel.java:128)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.AbstractFileChannel.write(AbstractFileChannel.java:206)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.AbstractFileChannel.write(AbstractFileChannel.java:192)
	at com.google.common@31.0-jre/com.google.common.io.ByteStreams.copy(ByteStreams.java:153)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.DavFolder.addMemberFile(DavFolder.java:94)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.DavFolder.addMember(DavFolder.java:70)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.doPut(AbstractWebdavServlet.java:713)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.AbstractNioWebDavServlet.doPut(AbstractNioWebDavServlet.java:129)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.execute(AbstractWebdavServlet.java:413)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.AbstractNioWebDavServlet.execute(AbstractNioWebDavServlet.java:95)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.service(AbstractWebdavServlet.java:316)
	at jetty.servlet.api@4.0.6/javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1619)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.MacChunkedPutCompatibilityFilter.doFilterHttp(MacChunkedPutCompatibilityFilter.java:51)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.AcceptRangeFilter.doFilterHttp(AcceptRangeFilter.java:37)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.MkcolComplianceFilter.doFilterHttp(MkcolComplianceFilter.java:40)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.PostRequestBlockingFilter.doFilterHttp(PostRequestBlockingFilter.java:39)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.UnicodeResourcePathNormalizationFilter.doFilterHttp(UnicodeResourcePathNormalizationFilter.java:53)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.LoggingFilter.doFilterHttp(LoggingFilter.java:32)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1594)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:506)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1571)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1372)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:463)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1544)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1294)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:192)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.Server.handle(Server.java:562)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:406)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:663)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:398)
	at org.eclipse.jetty.server@10.0.6/org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
	at org.eclipse.jetty.io@10.0.6/org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
	at org.eclipse.jetty.io@10.0.6/org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
	at org.eclipse.jetty.io@10.0.6/org.eclipse.jetty.io.SocketChannelEndPoint$1.run(SocketChannelEndPoint.java:101)
	at org.eclipse.jetty.util@10.0.6/org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
	at org.eclipse.jetty.util@10.0.6/org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
	at org.eclipse.jetty.util@10.0.6/org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
	at org.eclipse.jetty.util@10.0.6/org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:138)
	at org.eclipse.jetty.util@10.0.6/org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:378)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at java.base/java.nio.HeapByteBuffer.<init>(Unknown Source)
	at java.base/java.nio.ByteBuffer.allocate(Unknown Source)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkData.emptyWithSize(ChunkData.java:28)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkLoader.load(ChunkLoader.java:36)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkCache.loadChunk(ChunkCache.java:42)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkCache.lambda$get$0(ChunkCache.java:60)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkCache$$Lambda$1322/0x00000001006d22d0.call(Unknown Source)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4868)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3533)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2282)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2159)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2049)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache.get(LocalCache.java:3966)
	at com.google.common@31.0-jre/com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4863)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.fh.ChunkCache.get(ChunkCache.java:60)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.CleartextFileChannel.writeLockedInternal(CleartextFileChannel.java:159)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.CleartextFileChannel.writeLocked(CleartextFileChannel.java:128)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.AbstractFileChannel.write(AbstractFileChannel.java:206)
	at org.cryptomator.cryptofs@2.3.0/org.cryptomator.cryptofs.ch.AbstractFileChannel.write(AbstractFileChannel.java:192)
	at com.google.common@31.0-jre/com.google.common.io.ByteStreams.copy(ByteStreams.java:153)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.DavFolder.addMemberFile(DavFolder.java:94)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.DavFolder.addMember(DavFolder.java:70)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.doPut(AbstractWebdavServlet.java:713)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.AbstractNioWebDavServlet.doPut(AbstractNioWebDavServlet.java:129)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.execute(AbstractWebdavServlet.java:413)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.servlet.AbstractNioWebDavServlet.execute(AbstractNioWebDavServlet.java:95)
	at org.apache.jackrabbit.webdav.server.AbstractWebdavServlet.service(AbstractWebdavServlet.java:316)
	at jetty.servlet.api@4.0.6/javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
	at org.eclipse.jetty.servlet@10.0.6/org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1619)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.MacChunkedPutCompatibilityFilter.doFilterHttp(MacChunkedPutCompatibilityFilter.java:51)
	at org.cryptomator.frontend.webdav.servlet@1.2.0/org.cryptomator.webdav.core.filters.HttpFilter.doFilter(HttpFilter.java:21)

Just in case, my pc has 32 Gb RAM

Your problem is this: java.lang.OutOfMemoryError:
Your solution is this: WinFSP: How to use it

If you do not want to use WinSFT (=FUSE), you can also install and swich to Dokany instead. This should also solve your problem.

Thanks for the solution! But does it mean I have to install FUSE on other pc where I want to decrypt my files or it’s needed only for encryption purpose?

Neither nor.
FUSE is just a virtual file system (Filesystem in Userspace). Its how your decrypted files are presented in windows file explorer (simple saying).
Encryption and decryption is done by cryptomator app.
Thus, you can have on machine 1 a volume type dokany, on machine 2 volume type FUSE and on machine 3 volumetype WebDAV configured. Does not matter. But most likely you will run into your out of memory problems an all machines with similar configuration and WebDAV.
Furthermore, WebDAV has limitation when it comes to big files and regarding the available space on your virtual file system. So I would recommend to use one of these third party filesystems anyway.

1 Like