fs: enforce path search and unify inode permission helpers

[Abhishekmishra2808]

Summary

This change adds path-search permission checks (inode_checksearchpath) so every ancestor and mountpoint gate must allow X_OK, and unifies inode mode checks on inode_permission(). Mountpoint open no longer treats directory R/W bits as file open modes; traverse stays separate. An optional mountpt_operations.permission hook is added for in-volume DAC (tmpfs), without making VFS mount-crossing depend on it. Documentation describes the interface and mount-crossing behavior.

Impact

A private pseudoFS parent can no longer be bypassed by opening paths under a filesystem mounted beneath it. Non-root access gets EACCES when parent/mount traverse is denied. Behavior is unchanged when CONFIG_FS_PERMISSION is off. Filesystems that leave permission NULL are unaffected beyond the new VFS traverse checks on the mountpoint inode.

Testing

Non-root (euid=1000) denied access to a file under tmpfs mounted beneath a private (0700) pseudoFS directory - mount-crossing traverse enforcement.

abhishek@Lethallaptop:~/nuttx$ ./nuttx
nsh: mount: mount failed: 20

NuttShell (NSH) NuttX-13.0.0
nsh> id
uid=0 euid=0 gid=0 egid=0
nsh> mkdir /secure
nsh> mkdir /secure/mnt
nsh> mount -t tmpfs /secure/mnt
nsh> echo secret > /secure/mnt/a
nsh> chmod 0700 /secure
nsh> ls -l /secure
/secure:
 drwxrwxrwx       0       0          16 mnt/
nsh> echo 'root:x:0:0:/' > /tmp/ostest_passwd
nsh> echo 'testuser:x:1000:1000:/' >> /tmp/ostest_passwd
nsh> su testuser
nsh> id
uid=0 euid=1000 gid=0 egid=1000
nsh> cat /secure/mnt/a
nsh: cat: open failed: 13

Prints the secret (allowed under 0755).

nsh> mkdir /secure
nsh> mkdir /secure/mnt
nsh> mount -t tmpfs /secure/mnt
nsh> echo secret > /secure/mnt/a
nsh> chmod 0700 /secure
nsh> chmod 0755 /secure
nsh> echo 'root:x:0:0:/' > /tmp/ostest_passwd
nsh> echo 'testuser:x:1000:1000:/' >> /tmp/ostest_passwd
nsh> su testuser
nsh> id
uid=0 euid=1000 gid=0 egid=1000
nsh> cat /secure/mnt/a
secret

[github-actions]

MemBrowse Memory Report

arduino-mega2560

• flash: .text -6 B (-0.0%, 67,438 B / 262,144 B, total: 26% used)

esp32-devkitc

• ROM: .flash.rodata +4 B, .flash.text -12 B (-0.0%, 124,372 B / 4,194,272 B, total: 3% used)
• drom0_0_seg: .flash.rodata +4 B (+0.0%, 13,460 B / 4,194,272 B, total: 0% used)
• irom0_0_seg: .flash.text -12 B (-0.0%, 88,576 B / 3,342,304 B, total: 3% used)

hifive1-revb

• flash: .text +32 B (+0.0%, 83,412 B / 4,194,304 B, total: 2% used)

qemu-armv8a

• Code: .rodata +32 B, .text.inode_checkopenperm +32 B, .text.rename -24 B (+0.0%, 318,130 B)

qemu-intel64

• Code: .text -9 B (-0.0%, 8,657,985 B)

s698pm-dkit

• Code: .text +240 B (+0.1%, 363,600 B)

stm32-nucleo-f103rb

• flash: .text +32 B (+0.1%, 33,996 B / 131,072 B, total: 26% used)
  No memory changes detected for:
• mirtoo
• rx65n-rsk2mb

[acassis]

@Abhishekmishra2808 why does MemBrowse is reporting firmware increasing size even when CONFIG_DISABLE_MOUNTPOINT is disabled?

[Abhishekmishra2808]

@acassis , The growth is not from mount code. Those boards also typically leave CONFIG_FS_PERMISSION off, but we still always call the helpers from VFS (open / stat / rename / …). With permission disabled, they are stubs that return OK, yet the call sites + stub functions still land in .text

We can zero cost when CONFIG_FS_PERMISSION=n, and wrap those call sites in #ifdef CONFIG_FS_PERMISSION so the stubs disappear from the binary, right?

[acassis]

@acassis , The growth is not from mount code. Those boards also typically leave CONFIG_FS_PERMISSION off, but we still always call the helpers from VFS (open / stat / rename / …). With permission disabled, they are stubs that return OK, yet the call sites + stub functions still land in .text

We can zero cost when CONFIG_FS_PERMISSION=n, and wrap those call sites in #ifdef CONFIG_FS_PERMISSION so the stubs disappear from the binary, right?

Yes, I think removing these stubs when FS_PERMISSION is disabled it the right approach

[Abhishekmishra2808]

@acassis PTAL

[acassis]

Thank you very much @Abhishekmishra2808 !