macOS uses inode number 1 on exFAT #585

ghost · 2021-07-27T08:16:25Z

As referenced in #556, starting from this comment #556 (comment) I hijacked the issue to discover that macOS uses the inode number 1 on an exFAT filesystem.

This leads to the error message
panic: using reserved ID 1 for inode number
when using gocryptfs later than v1.8.0.

Regarding @rfjakob's request

I would just try and see what
rm foo
touch foo
ls -li foo
says on exfat.

$> rm foo # make sure foo doesn't exist
rm: foo: No such file or directory
$> touch foo
$> ls -li foo
1 -rwxrwxrwx  1 user  group  0 27 Jul 10:04 foo

Interestingly:

$> touch foo2
$> ls -li foo*
1 -rwxrwxrwx  1 user  group  0 27 Jul 10:04 foo
1 -rwxrwxrwx  1 user  group  0 27 Jul 10:06 foo2

But that's probably some optimization, because:

$> rm foo foo2
$> echo "text" >> foo
$> touch foo2
$> ls -li foo*
5163394 -rwxrwxrwx  1 user  group  5 27 Jul 10:08 foo
      1 -rwxrwxrwx  1 user  group  0 27 Jul 10:08 foo2

The problem with macOS is probably, that when opening a directory in the Finder, the Finder immediately places a file called .DS_Store in that directory.
This file stores things like how the folder's contents should be displayed, such as list, details or symbols, and in the case of the latter, the specific positions of these symbols.

Initially this file is empty, and thus probably gets inode number 1, once the directory is opened for the first time in the Finder.

Thus: When one only works with directories on the command line and never creates empty files, this issue is unlikely to hit anybody.
The question remains, why gocryptfs v1.8.0 is unaffected.
Probably because of go-fuse?

Edit: Interestingly this "empty file inode number 1 sharing" does not happen on macOS's native APFS:

$> touch foo foo2
$> ls -li foo*
36977462 -rw-r--r--  1 user  group  0 27 Jul 10:30 foo
36977463 -rw-r--r--  1 user  group  0 27 Jul 10:30 foo2

The text was updated successfully, but these errors were encountered:

rfjakob · 2021-07-27T09:32:59Z

$> touch foo2
$> ls -li foo*
1 -rwxrwxrwx  1 user  group  0 27 Jul 10:04 foo
1 -rwxrwxrwx  1 user  group  0 27 Jul 10:06 foo2

Oh dear that's not good

rfjakob · 2021-07-27T09:36:33Z

Can you run statfs ( https://github.com/rfjakob/gocryptfs/tree/master/contrib/statfs , run "go build" there and you get the executable) on foo ? I need to get the filesystem id for exfat so I can detect it on startup.

ghost · 2021-07-27T09:44:58Z

This should be the relevant output for exFAT:

"Fsid": {
	"Val": [
		16777233,
		35
	]
},
"Owner": 0,
"Type": 35,
"Flags": 2101784,
"Fssubtype": 0,
"Fstypename": [
	101,
	120,
	102,
	97,
	116,
	0,
	0,
	0,
	0,
	0,
	0,
	0,
	0,
	0,
	0,
	0
],

Just for completion, this is the output for APFS

    "Fsid": {
            "Val": [
                    16777224,
                    26
            ]
    },
    "Owner": 0,
    "Type": 26,
    "Flags": 76582912,
    "Fssubtype": 1,
    "Fstypename": [
            97,
            112,
            102,
            115,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0
    ],

rfjakob · 2021-07-28T10:00:32Z

Perfect, thanks! Could you try passing -sharedstorage on mounting? This should work around the issue, as it disables hard links.

ghost · 2021-07-30T18:03:20Z

This worked.

Now gocryptfs v2.0.1 has no problems anymore, even though on the exFAT disk there are still files with inode id 1.
For reference, I'm still using FUSE 3.11.2.

Thank you.

What are the downsides of this?

rfjakob · 2021-07-31T15:52:17Z

Downside is that metadata caching is disabled, leading to slower operation. Second downside is that this workaround probably stopped working with 1bc1db6 just now.

The root directory always get nodeid 1, and the v2 api originally had "nodeid == inode number". But since 1d0096e, inode number and node id are independent, and we can allow inode number 1. Why does this matter? MacOS actually uses inode number 1 on ExFAT mounts, triggering a panic on overlay filesystems like gocryptfs: panic: using reserved ID 1 for inode number Fixes: rfjakob/gocryptfs#585 Change-Id: I5e3476da65400d32efdb3dac1fe4901c64c00a89

Let's not crash anymore when we see inode number 1 ( hanwen/go-fuse@0aaef6d ) #585

rfjakob · 2021-08-02T18:07:50Z

Ok, final workaround merged just now. Could you test?

You should see

The message ExFAT detected, disabling hard links when mounting
Stable inode numbers
inode number 1 passed through to the plaintext folder

ghost · 2021-08-05T15:09:31Z

c3c9513 works like a charm now, except that I didn't see that message.

rfjakob · 2021-08-05T16:21:56Z

Hmm the missing message worries me a bit, but maybe it went to syslog. Could you try again with -nosyslog?

ghost · 2021-08-05T16:38:36Z

Here:

$> gocryptfs -fusedebug -fg -d -nosyslog /Volumes/T7-0-data1/backup-data/ ./plain/
	cli args: ["gocryptfs" "-fusedebug" "-fg" "-d" "-nosyslog" "/Volumes/T7-0-data1/backup-data/" "./plain/"]
	Password: 
	Decrypting master key
	cryptocore.New: key=32 bytes, aeadType=BackendGoGCM, IVBitLen=128, useHKDF=true, forceDecode=false
	contentenc.New: plainBS=4096, forceDecode=false
	CryptoCore.Wipe: Only nil'ing stdlib refs
	cryptocore.New: key=32 bytes, aeadType=BackendGoGCM, IVBitLen=128, useHKDF=true, forceDecode=false
	contentenc.New: plainBS=4096, forceDecode=false
	nametransform.New: longNames=true, raw64=true, badname=[]
	frontendArgs: {
		"Cipherdir": "/Volumes/T7-0-data1/backup-data",
		"PlaintextNames": false,
		"LongNames": true,
		"PreserveOwner": false,
		"ForceOwner": null,
		"ConfigCustom": false,
		"NoPrealloc": false,
		"SerializeReads": false,
		"ForceDecode": false,
		"Exclude": null,
		"ExcludeWildcard": null,
		"ExcludeFrom": null,
		"Suid": false,
		"KernelCache": false,
		"SharedStorage": false
	}
	18:29:41.391680 rx 2: INIT n0 {7.19 Ra 0x200000 CASE_INSENSITIVE,XTIMES,VOL_RENAME,ATOMIC_O_TRUNC,0x18000000} 
	18:29:41.391811 tx 2:     OK, {7.8 Ra 0x200000  0/0 Wr 0x20000 Tg 0x0}
	18:29:41.391965 rx 3: STATFS n1 
	18:29:41.391985 rx 2: STATFS n1 
	18:29:41.392055 tx 2:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.392055 tx 3:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.392527 rx 3: STATFS n1 
	18:29:41.392540 rx 2: STATFS n1 
	18:29:41.392604 tx 2:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.392609 tx 3:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.393578 rx 3: STATFS n1 
	18:29:41.393661 tx 3:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.394177 rx 2: STATFS n1 
	18:29:41.394223 tx 2:     OK, {blocks (2607424,2607424)/15261345 files 2607424/15261345 bs131072 nl0 frs131072}
	18:29:41.394318 rx 3: ACCESS n1 {u=501 g=20 x} 
	18:29:41.394566 tx 3:     OK
	18:29:41.394843 rx 2: LOOKUP n1 [".go-fuse-epoll-hack"] 20b
	18:29:41.394920 tx 2:     OK, {n18446744073709551615 g0 tE=0s tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395049 rx 3: GETATTR n18446744073709551615  
	18:29:41.395091 tx 3:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395210 rx 2: GETATTR n18446744073709551615  
	18:29:41.395289 tx 2:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395434 rx 3: ACCESS n18446744073709551615 {u=501 g=20 w,r} 
	18:29:41.395443 tx 3:     OK
	18:29:41.395522 rx 2: GETATTR n18446744073709551615  
	18:29:41.395565 tx 2:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395629 rx 3: OPEN n18446744073709551615 {RDWR} 
	18:29:41.395640 tx 3:     OK, {Fh 18446744073709551615 }
	18:29:41.395779 rx 2: SETATTR n18446744073709551615 {size 0, fh 18446744073709551615} 
	18:29:41.395809 tx 2:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395863 rx 3: GETATTR n18446744073709551615  
	18:29:41.395894 tx 3:     OK, {tA=0s {M0100644 SZ=0 L=1 0:0 0 0:18446744073709551615 A 0.000000 M 0.000000 C 0.000000}}
	18:29:41.395977 rx 2: FLUSH n18446744073709551615 {Fh 18446744073709551615} 
	18:29:41.396018 tx 2:     OK
	18:29:41.396080 rx 3: RELEASE n18446744073709551615 {Fh 18446744073709551615 RDWR  L0} 
	18:29:41.396089 tx 3:     OK
	Filesystem mounted and ready.

I tried with -sharedstorage, too, but it doesn't show up there either.

When exactly is the message expected to appear?

rfjakob · 2021-08-05T16:40:58Z

Message is:

detectQuirks: ExFAT detected, disabling hard links. See https://github.com/rfjakob/gocryptfs/issues/585 for why.

ghost · 2021-08-05T16:43:32Z

Just in case, you need the statfs again (performed on the encrypted directory):

$> ./statfs /Volumes/T7-0-data1/backup-data/ | head -n 40
{
	"Bsize": 131072,
	"Iosize": 131072,
	"Blocks": 15261345,
	"Bfree": 2607424,
	"Bavail": 2607424,
	"Files": 15261345,
	"Ffree": 2607424,
	"Fsid": {
		"Val": [
			16777233,
			40
		]
	},
	"Owner": 0,
	"Type": 40,
	"Flags": 2101784,
	"Fssubtype": 0,
	"Fstypename": [
		101,
		120,
		102,
		97,
		116,
		0,
		0,
		0,
		0,
		0,
		0,
		0,
		0,
		0,
		0,
		0
	],
	"Mntonname": [
		47,
		86,
		111,

rfjakob · 2021-08-05T16:45:13Z

Oh dear, "Type" is 40 now. It was 35 in your earlier post

ghost · 2021-08-05T16:51:13Z

That is really strange.
I haven't changed anything and copied the command from the bash history. O.o

ghost · 2021-08-05T16:55:08Z

Oh, maybe:
I updated from macOS 11.4 to macOS 11.5.1 one day after the post with the 35.

rfjakob · 2021-08-05T17:04:26Z

https://stackoverflow.com/questions/46345265/macos-correct-programmatic-way-to-determine-volume-info

We need to look at f_fstypename acc. to https://stackoverflow.com/a/52299141/1380267 : > As filesystem type numbers are now assigned at runtime in > recent versions of MacOS, you must use f_fstypename to > determine the type. #585

rfjakob · 2021-08-11T18:26:00Z

@ueberbelichtetesfoto could you try again with latest master? I check f_fstypename now, which should be stable, but, lacking a Mac, could not test it.

ghost · 2021-08-11T20:02:27Z

Will take some time. Next time I can test will be this weekend.

ghost · 2021-08-16T14:12:01Z

I think this looks as you wanted it to be:

$> gocryptfs --version
gocryptfs v2.0.1-49-g2d386fc.HEAD without_openssl; go-fuse v2.1.1-0.20210802120645-15a8bb029a4e; 2021-08-16 go1.16.3 darwin/amd64

$> gocryptfs /Volumes/T7-0-data1/backup-data/ ./plain/
Password: 
Decrypting master key
DetectQuirks: ExFAT detected, disabling hard links. See https://github.com/rfjakob/gocryptfs/issues/585 for why.
Filesystem mounted and ready.

Also creating and dealing with empty files works just fine.
They get inode number 1 in both, the underlying ciphertext directory and the overlaying plaintext directory.

Edit: Creating two empty files leads to the two files sharing the inode number 1.
Is this expected?
Should I still use -sharedstorage?

rfjakob · 2021-08-16T14:45:49Z

I think this looks as you wanted it to be:

Yes looks good, thanks for testing!

Edit: Creating two empty files leads to the two files sharing the inode number 1.
Is this expected?

Yes, they internally get a different identifier called "generation number", but to the user only the inode number is visible.

Should I still use -sharedstorage?

Not needed, no!

rfjakob added a commit that referenced this issue Aug 2, 2021

go mod: upgrade go-fuse

75cf36f

Let's not crash anymore when we see inode number 1 ( hanwen/go-fuse@0aaef6d ) #585

rfjakob closed this as completed in c3c9513 Aug 2, 2021

rfjakob reopened this Aug 11, 2021

rfjakob closed this as completed Aug 16, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macOS uses inode number 1 on exFAT #585

macOS uses inode number 1 on exFAT #585

ghost commented Jul 27, 2021 •

edited by ghost

Loading

rfjakob commented Jul 27, 2021

rfjakob commented Jul 27, 2021

ghost commented Jul 27, 2021

rfjakob commented Jul 28, 2021

ghost commented Jul 30, 2021 •

edited by ghost

Loading

rfjakob commented Jul 31, 2021 •

edited

Loading

rfjakob commented Aug 2, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021 •

edited

Loading

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

ghost commented Aug 5, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

rfjakob commented Aug 11, 2021 •

edited

Loading

ghost commented Aug 11, 2021

ghost commented Aug 16, 2021 •

edited by ghost

Loading

rfjakob commented Aug 16, 2021

macOS uses inode number 1 on exFAT #585

macOS uses inode number 1 on exFAT #585

Comments

ghost commented Jul 27, 2021 • edited by ghost Loading

rfjakob commented Jul 27, 2021

rfjakob commented Jul 27, 2021

ghost commented Jul 27, 2021

rfjakob commented Jul 28, 2021

ghost commented Jul 30, 2021 • edited by ghost Loading

rfjakob commented Jul 31, 2021 • edited Loading

rfjakob commented Aug 2, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021 • edited Loading

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

ghost commented Aug 5, 2021

ghost commented Aug 5, 2021

rfjakob commented Aug 5, 2021

rfjakob commented Aug 11, 2021 • edited Loading

ghost commented Aug 11, 2021

ghost commented Aug 16, 2021 • edited by ghost Loading

rfjakob commented Aug 16, 2021

ghost commented Jul 27, 2021 •

edited by ghost

Loading

ghost commented Jul 30, 2021 •

edited by ghost

Loading

rfjakob commented Jul 31, 2021 •

edited

Loading

rfjakob commented Aug 5, 2021 •

edited

Loading

rfjakob commented Aug 11, 2021 •

edited

Loading

ghost commented Aug 16, 2021 •

edited by ghost

Loading