Some files not readable on High Sierra, but works fine in the iOS app

os:mac
os:ios

#1

I tried opening the files with Word and nothing happened. Copying and other actions result in Finder error 43 or 36 and trying to copy with Unix cp returns “Interrupted System Call” after these actions the files disappear from the file system until I restart Cryptomator. In the iOS app I can open these files without any issues.

Here is a debug log from trying to copy a document. Is the 404 Not Found Response returned from Cryptomator?


12:17:36.038 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - REQUEST for //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx on HttpChannelOverHttp@20667b3c{r=12,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx}
GET //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx HTTP/1.1
Host: localhost:53710
Accept: */*
User-Agent: WebDAVFS/3.0.0 (03008000) Darwin/17.5.0 (x86_64)
Connection: keep-alive


12:17:36.038 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} onContentComplete
12:17:36.038 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} onRequestComplete
12:17:36.039 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} handle //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx 
12:17:36.039 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=false,a=DISPATCHED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} action DISPATCH
12:17:36.039 [Server thread 007] TRACE o.c.f.webdav.servlet.LoggingFilter - REQUEST 133:
GET /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx HTTP/1.1
User-Agent: WebDAVFS/3.0.0 (03008000) Darwin/17.5.0 (x86_64)
Connection: keep-alive
Host: localhost:53710
Accept: */*

12:17:36.053 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=DirectByteBuffer@27ed2ec1[p=0,l=397,c=32768,r=397]={<<<<html>\n<head>\n<me.../body>\n</html>\n>>>1)\x91\xF2O\xBd1\x17%W`1\x8cS!\x89\x93...JWb\x1eq\x14E\xD8(\xDe\xA3\x01A\xC9\xC6} complete=true committing=true callback=Blocker@48091889{null}
12:17:36.054 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - COMMIT for /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx on HttpChannelOverHttp@20667b3c{r=12,c=true,a=DISPATCHED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx}
404 Not Found HTTP/1.1
Date: Fri, 06 Apr 2018 10:17:36 GMT
Accept-Ranges: bytes
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1


12:17:36.054 [Server thread 007] TRACE o.c.f.webdav.servlet.LoggingFilter - RESPONSE 133:
404
Accept-Ranges: bytes
Cache-Control: must-revalidate,no-cache,no-store
Date: Fri, 06 Apr 2018 10:17:36 GMT
Content-Type: text/html;charset=iso-8859-1

12:17:36.054 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=true,a=COMPLETING,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} action COMPLETE
12:17:36.054 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - COMPLETE for /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx written=397
12:17:36.054 [Server thread 007] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=12,c=false,a=IDLE,uri=null} handle exit, result COMPLETE
12:17:36.058 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - REQUEST for //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx on HttpChannelOverHttp@20667b3c{r=13,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx}
PROPFIND //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx HTTP/1.1
Host: localhost:53710
Content-Type: text/xml
Depth: 0
Accept: */*
User-Agent: WebDAVFS/3.0.0 (03008000) Darwin/17.5.0 (x86_64)
Content-Length: 179
Connection: keep-alive


12:17:36.058 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=false,a=IDLE,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} handle //localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx 
12:17:36.059 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=false,a=DISPATCHED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} action DISPATCH
12:17:36.059 [Server thread 005] TRACE o.c.f.webdav.servlet.LoggingFilter - REQUEST 134:
PROPFIND /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx HTTP/1.1
Accept: */*
User-Agent: WebDAVFS/3.0.0 (03008000) Darwin/17.5.0 (x86_64)
Connection: keep-alive
Host: localhost:53710
Content-Length: 179
Depth: 0
Content-Type: text/xml

12:17:36.060 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - sendResponse info=null content=HeapByteBuffer@611623db[p=0,l=0,c=0,r=0]={<<<>>>} complete=true committing=true callback=Blocker@48091889{null}
12:17:36.060 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - COMMIT for /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx on HttpChannelOverHttp@20667b3c{r=13,c=true,a=DISPATCHED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx}
404 Not Found HTTP/1.1
Date: Fri, 06 Apr 2018 10:17:36 GMT


12:17:36.061 [Server thread 005] TRACE o.c.f.webdav.servlet.LoggingFilter - RESPONSE 134:
404
Date: Fri, 06 Apr 2018 10:17:36 GMT

12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=true,a=COMPLETING,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} action COMPLETE
12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - COMPLETE for /U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx written=0
12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=true,a=COMPLETED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} onContent Content@19056271{HeapByteBufferR@5491744d[p=281,l=460,c=8192,r=179]={PROPFIND /U4zLOJf... keep-alive\r\n\r\n<<<<?xml version="1....\n</D:propfind>\n>>>6\x0c8\x08\x11\x01,\tXcomments...n\xD4\x12\x10\x11\x0f\x18;\x0c=\x08\x11\x01,\t}}
12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=true,a=COMPLETED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} onContentComplete
12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=true,a=COMPLETED,uri=//localhost:53710/U4zLOJfFTDM0/DropboxEncrypted/Sonstige%20Schreiben/Ku%CC%88ndigung%20Telekom.docx} onRequestComplete
12:17:36.061 [Server thread 005] DEBUG org.eclipse.jetty.server.HttpChannel - HttpChannelOverHttp@20667b3c{r=13,c=false,a=IDLE,uri=null} handle exit, result COMPLETE


#2

Yes, it is. Ku%CC%88ndigung%20Telekom.docx is getting my attention. Especially %CC%88

@overheadhunter: If I’m not mistaken, u%CC%88 is ü encoded with NFD. But shouldn’t it have been normalized to NFC? Or am I mixing it up?

@kai: Could you please try and rename this file with the iOS app and remove the ü. After Dropbox has finished syncing on your Mac, could check if you’re now able to access the file?


#3

Indeed removing the ü on iOS fixes the problem. I can now open and copy the file on my mac. I already use Cryptomator for a long time and did not have problems with these special characters before however. In fact I can rename it back to Kündigung Telekom.docx on my mac and it works. Perhaps some version of Cryptomator or OSX broke the encoding? I have multiple files affected by this and going through all directories manually on the iOS app and to change the file names would take a lot of time. Any ideas?


#4

We have trouble reproducing the issue. We tried various versions of Cryptomator and various encodings. The files were always accessible.

  • What version of macOS are you using? Edit: Just seen that you’ve said High Sierra in the title.
  • What version of Cryptomator for macOS are you currently using?
  • When was the last time the affected files were normally accessible? If you remember, do you also remember which Cryptomator version you’ve used? Related: Do you update Cryptomator regularly or have you updated recently?
  • Were other operating systems involved other than macOS and iOS?
  • What is the origin of the files? Did you create them with macOS or iOS? Or are they maybe created with Windows?

#5

macOS 10.13.4, Cryptomator 1.3.2

I use these files very rarely (more like an archive) and I am not sure when I opened them the last time. I cannot rule out that I never opened them via Cryptomator before actually.

Before this issue appeared I did not even have the iOS app so I only used Cryptomator on macOS with Dropbox.

I had windows computers a long time ago and I am not 100% sure, whether I created the affected files with Windows based Word or not macOS based word.

Generally, I updated Cryptomator immediately when prompted after application start.

I found one affected PDF that was generated by a scanner / camera with an Ü from 2014 that I just moved to Cryptomator when I started using it and probably never opened again. I found the original PDF from the email, downloaded it from the email server and copied to Cryptomator. After syncing with Dropbox and restarting Cryptomator it was still fine. However, the old copy of this PDF that moved to Cryptomator, when I started using it, has the same issue like I said. Maybe this indicates that this encoding error appeared in an older combination of macOS and Cryptomator, but not current versions?


#6

Thank you for the info!

This sounds plausible and could be the reason why it’s not reproducible anymore.

Sooo, we actually reproduced it in the meantime by “simulating” the issue. We basically renamed the ciphertext names directly and forced the cleartext to be encoded with NFD.

We’re going to update Sanitizer so that it can solve this specific issue, too. And we have to find a solution for Cryptomator itself as well but this will take more time.

I’ll let you know when the updated version of Sanitizer is available. It’s probably going to be version 0.15.


#7

Created an issue on GitHub:


#8

@kai: We’ve updated Sanitizer to v0.15 and added detection and solution of name normalization problems.

We’ve also updated the Sanitizer: How to Use guide.

This is the command that you’re going to run in Terminal to solve name normalization problems.

java -jar /path/to/sanitizer-0.15.jar check -vault /path/to/vault -solve NameNormalization

#9

I was a bit busy, but now I run the Sanitizer and it seems to work now. The sanitizer also informed me about EmptyEncryptedFile and OrphanMFile. Is there any documentation about what exactly this is and if I need to worry about it?


#10

Unfortunately, there is no documentation but here is some info on these two problems:

EmptyEncryptedFile
It means that a 0 byte cleartext file (88 byte ciphertext file since the header is 88 bytes long) has been found. It doesn’t really have to mean anything because it’s totally valid to encrypt an empty cleartext file. That’s why it’s just informative.

OrphanMFile
It means that a file inside the directory m was found that doesn’t have a corresponding file in the directory d. m files are metadata files that are created for name shortening. If a filename is too long, such file will be created. Since Cryptomator doesn’t delete any m files even though they’re not needed anymore, you’ll get this information. These files cause no harm.