1. 59

  2. 3

    This is amazing. Linux maintainers have been working on NTFS support for basically 20 years, but with no support or documentation from Microsoft, it seemed impossible. There was even a project called Captive NTFS which aimed to use the actual Windows kernel NTFS implementation in a Linux driver with a hellishly complicated compatibility layer. Hopefully the new Linux driver will be useful for other OSes and tools like backup and disk recovery utilities as well.

    1. 3

      I’m confused… Haven’t we had great NTFS support for years?

      1. 28

        More context: https://arstechnica.com/gadgets/2021/08/paragon-is-working-to-get-its-ntfs3-filesystem-into-the-linux-kernel/

        Both existing implementations have problems, however. The in-kernel implementation of NTFS is extremely old, poorly maintained, and should only be used read-only. As a result, most people who actually need to mount NTFS filesystems on Linux use the ntfs-3g driver instead.

        Ntfs-3g is in reasonably good shape—it’s much newer than the in-kernel ntfs implementation, and as Linux filesystem guru Ted Ts’o points out, it actually passes more automated filesystem tests than Paragon’s own ntfs3 does.

        Unfortunately, due to operating in userspace rather than in-kernel, ntfs-3g’s performance is abysmal. In Ts’o’s testing, Paragon’s ntfs3 completed automated testing in 8,106 seconds—but the FUSE-based ntfs-3g required a whopping 34,783 seconds.

        (In summary, what the other commenter said.)

        1. 17

          The kernel driver has never had stable read-write support. The FUSE driver has been reasonably stable but is quite slow.

        2. 2

          Has anyone considered trying NTFS root filesystem yet? It might be an…. interesting alternative to partitioning for dual boots.

          1. 3

            I’m fairly certain it’s not possible due to different features between the filesystems - in particular no suid means sudo won’t work. I’m also not sure mapping to different users on Linux works properly, though I haven’t checked in a while.

            1. 1

              That can probably bevworked around with creative use of extended attributes, if someone really wants to do it.

              1. 1

                I’m pretty sure NTFS has something for setuid, since Interix supported it.

                1. 9

                  NTFS is a lot like BeFS: the folks talking to the filesystem team didn’t provide a good set of requirements early on and so they ended up with something incredibly general. NTFS, like BeFS, is basically a key-value store, with two ways of storing values. Large values can (as with BeFS) be stored in disk blocks, small values are stored in a reserved region that looks a little bit like a FAT filesystem (BeFS stores them in the inode structure for the file).

                  Everything is layered on top of this. Compression, encryption, and even simple things like directories, are built on top of the same low-level abstraction. This means that you can take a filesystem with encryption enabled and mount it with an old version of NT and it just won’t be able to read some things.

                  This is also the big problem for anything claiming to ‘support NTFS’. It’s fairly easy to support reading and writing key-value pairs from an NTFS filesystem but full support means understanding what all of the keys mean and what needs updating for each operation. It’s fairly easy to define a key-value pair that means setuid, but if you’re dual booting and Windows is also using the filesystem then you may need to be careful to not accidentally lose that metadata.

                  I also don’t know how the NTFS driver handles file ownership and permissions. In a typical *NIX filesystem, you have an small integer UID combined with a two-byte bitmap of permissions. You may also have ACLs, but they’re optional. In contrast, NTFS exposes owners as UUIDs (much larger than a uid that any *NIX program understands) and has only ACLs (which are not expressed with the same verbs as NFSv4 or POSIX ACLs), so you need some translation layer and need to be careful that this doesn’t introduce incompatibilities with the Windows system.

                  You’re probably better off creating a loopback-mounted ext4 filesystem as a file in NTFS and just mounting the Windows home directory, if you want to dual boot and avoid repartitioning.

                  Note that WSL1 uses NTFS and provides Linux-compatible semantics via a filter driver. If someone wants to reverse engineer how those are stored (wlspath gives the place they live in the UNC filesystem hierarchy) then you could probably have a Linux root FS that uses the same representation as WSL and also uses the same place in the UNC namespace so that Windows tools know that they’re special.

                  1. 1

                    What is used by WSL 2?

                    1. 5

                      WSL2 is almost totally unrelated to WSL, it’s a Linux VM running on Hyper-V (I really wish they’d given WSL2 a different name). Its root FS is an ext4 block device (which is backed by a file on the NTFS file system). Shared folders are exported as 9p-over-VMBus from the host.

                      This is why the performance characteristics of WSL and WSL2 are almost exactly inverted. WSL1 has slow root FS access because it’s an NTFS filesystem with an extra filter driver adding POSIX semantics but the perf accessing the Windows FS is the same because it’s just another place in the NTFS filesystem namespace. WSL2 has fast access to the root FS because it’s a native Linux FS and the Linux VM layer is caching locally, but has much slower access to the host FS because it gets all of the overhead of NTFS, plus the overhead of serialising to an in-memory 9p transport, plus all of the overhead of the Linux VFS layer on top.

                      Hopefully at some point WSL will move to doing VirtIO over VMBus instead of 9p. The 9p filesystem semantics are not quite POSIX and the NTFS semantics are not 9p or POSIX, so you have two layers of impedance mismatch. With VirtIO over VMBus, the host could use the WSL interface to the NTFS filesystem and directly forward operations over a protocol that uses POSIX semantics.

                      There are some fun corner cases in the WSL filesystem view. For example, if you enable developer mode then ln -s in WSL will create an NTFS symbolic link. If you disable developer mode then unprivileged users aren’t allowed to create symbolic links (I have no idea why) and so WSL creates an NTFS junction. Nothing on the system other than WSL knows what to do with a junction that refers to a file (the rest of Windows will only ever create junctions that refer to directories) and so will report the symlink as a corrupted junction. This is actually a pretty good example of the split between being able to store key-value pairs and knowing what they mean in NTFS: both WSL and other Windows tools use the same key to identify a junction but WSL puts a value in that nothing else understands.

                      1. 1

                        Actual Linux filesystems. Because it’s just a Linux kernel, in Hyper-V, with dipping mustards.

                2. 2

                  Why not go the other way around and boot Windows off of btrfs? :D

                  1. 1

                    This is only a proof of concept at this stage - don’t use this for anything serious.

                    But really, why not, you have backups… right? :P