1. 49

  2. 13

    Excellent article! Some other platforms that have interesting approaches to file storage:

    • VMS; Files-11/ODS heavily inspired NTFS. There’s versions (DEC stuff generally had this, and probably where the Alto got it), records (structure in files), and a funky path syntax.
    • Anything mainframe; highlights include that in the very old days, there was a very direct path to storage and structure to match (manifested in things like CKD keys with variable length blocks).
    • Lisp machines and Smalltalk; due to their focus on image-based worlds where the objects are just stored in RAM and stored on disk the same way through paging, usually.
    • (You know I was going to mention this) IBM i; perhaps the most commercially successful implementation of single-level storage (in one of the more radical implementations, memory is another level of cache, and disk is where the single virtual address space lives) and capabilities. It uses these primitives to build on top an entire relational database.
    1. 11

      VMS and a lot of contemporary systems didn’t really have a notion of directories at the filesystem level, they had a convention that a particular character or character sequence was an organisational separator and had efficient search for all files with a common prefix. I think it’s worth differentiating between folders (a UI pattern) and directories (a filesystem lookup optimisation pattern), even though one is typically used to implement the other. A folder is a human-visible organisational unit for identifying a group of files. A directory is a way of pruning the search space when examining part of the filesystem.

      This distinction was particularly apparent in BeFS, where a directory is just the cached result of a search. When you update any metadata on a file in BeFS, you also update any directories that are defined by a query on that metadata. If you store your music on BeFS, this lets you have a directory tree of {artist}/{album}/{song} or {genre}/{song} or whatever. Importantly, it broke the idea that directories define a strict hierarchy. HCI studies have shown that around 10% of the population naturally thinks in terms of hierarchies for organisation (I’d love to see someone do some research on the fact that this is roughly the same proportion that find learning programming easy and that most programming languages have strong hierarchies in their structure). This was very apparent when iTunes came out: programmers who already organised their music in a clean hierarchy hated it, the general population that thought in terms of arbitrary filters on their music loved it. BeOS captured that distinction for all filesystem objects.

      The notion that a filesystem is a DAG is baked into POSIX quite deeply. A lot of POSIX tools expect to be able to do tree walks on a filesystem and exhaustively search it and so POSIX disallows hard links on directories. XNU with HFS+ or APFS allows directory hard links but with a carefully restricted system permission that is granted only to the Time Machine daemon, so that it can do file-granularity deduplication across incremental backups (each backup is a complete directory tree but any directory that hasn’t changed since the last backup is a hard link to the same directory in the previous backup).

      It’s worth noting that some of the features from the article are coming back. CUSE on Linux / *BSD allows you do implement Hydra-like files: they show up as device nodes and not regular files, but most *NIX utilities don’t check that they’re talking to a file and not a device node. FUSE lets you implement entire filesystems with callback hooks but doesn’t let you mix and match files from your FUSE FS with regular files unless you write a pass-through to another FS or use some kind of union FS.

      FUSE (and IFS on Windows) has also been used to implement some of the other things the article talks about, such as incremental restore. A lot of cloud storage things present a FUSE / IFS filesystem and only download files when they’re accessed. Everyone is slowly reinventing AFS.

      Capsicum layers a capability model over POSIX. File descriptors are capabilities and programs in capability mode do not have access to the global namespace. They can openat given a filesystem path relative to a directory descriptor, but they can’t open arbitrary files.

    2. 4

      As I commented on the blog post: the Xerox Alto had many different OSs written for it. The filesystem described in the article is from one of them, but I didn’t scan the paper thoroughly enough to figure out which one. It was written in BCPL, which means it was probably fairly early since later work was mostly done in Mesa.

      1. 4
        1. 2

          Newton OS had a lot of clever ideas that I wish were revisited.

          1. 3

            The one that bugs me the most: copy and paste. On the Newton, you selected something and dragged it to the edge of the screen. It then gets attached as a little label. You can switch application and drag it back. This is, hands down, the nicest way I’ve seen of transferring snippets between full-screen applications on a touchscreen device. I think Apple even had a patent on this that was still valid when the first iPhone came out, yet it shipped without copy and paste because Steve Jobs hated the Newton. I hoped that, after he died, they’d reintroduce this, but they didn’t and the copy-and-paste support on iOS is still unbelievable clunky in comparison to the Newton.

            1. 2


          2. 2

            Off topic, but… I’m not a font nerd, I don’t generally notice or care about them much, and the font used in that website is beautiful. Especially the italic. Apparently it’s called Lora, I’m pretty sure I’ve never seen it before.

            1. 1
              1. 1