A way that I’ve been thinking about this is “early binding vs late binding”. The Alan Kay quote is “OOP to me means only messaging, local retention, and protection and hiding of state-process, and extreme late-binding of all things.” So he’s very pro-late binding. But early binding (can) produce more efficient and robust systems, at least theoretically. Going through before hand, making sure all bindings are correct and then optimizing them should give you a faster, more reliable system. At the very least, it can’t hurt. On the other hand, the reason Kay likes late binding is that the system is better for building new things if all the parts are bound at the last moment.
This applies fractally:
Dynamic linking vs static linking of libraries
Importing multiple JavaScript CDN URLs to an HTML page vs hard coding a single hashed treeshaken bundle
Docker files vs Nix packages
In Go parlance, using an interface vs using a concrete type
Using a dynamically typed language vs statically typed
Using an interpreted language vs a compiled language
Writing a shell script vs distributing a binary that does the same thing
Using an Excel file vs using an enterprise accounting SaaS
Having a fleet of Kubernetes microservices vs having a single big iron physical server running a monolith
Communicating to a device via Wifi + HTTP vs a dedicated hardlink with binary protocol
Dynamic sacrifices performance and reliability in exchange for ease of continued development (source is available on disk; types are loose). In cases where performance is well passed good enough, this seems like a good tradeoff.
I think what the “narrow waist” concept adds is a description of when the advantages of the dynamic system are likely to compound rather than the liabilities. VBScript was dynamic, but it quickly became an unmaintainable mess. Having a big commercial website that links together 6 separate PHP and Java backends via an Enterprise Service Bus is going to bad ugly and slow, but it will work because the ESB is a narrow waist.
Yeah that’s is fair. I don’t use the term that much (maybe because almost everything I do is late bound), but wikipedia says it’s synonymous with “dynamic binding”:
In other words, a name is associated with a particular operation or object at runtime, rather than during compilation.
I just call that “dynamic”, so it probably depends on your personal history of problems and domains. I’m more focused on systems, whereas others might be more focused on GUIs as an application of OO.
I think what the “narrow waist” concept adds is a description of when the advantages of the dynamic system are likely to compound rather than the liabilities.
Well I’d say the narrow waist is first about interoperability and economy of code. And that does seem to come with a lot of dynamic mechanisms, as with shell, Unix, the web, and distributed systems.
So yeah the post mixes many topics. It started out with “narrow waists”, but then I could not help addressing the dynamic vs. static styles. I added the #1 and #2 dichotomy in the intro toward the very end. It was something of a surprising relationship but it does appear to hold up after looking at all the examples, the history of the biggest evolved systems in software.
There are some examples of static narrow waists – as mentioned LLVM, and STL design. Not sure what to do with those – maybe I should “define them out” of what I’m making claims about.
I feel like the narrow waist is a symptom not a cause. The cause is trying for orthogonality (hearkening back to the goals of ALGOL 68). For example, in the STL we explicitly get NxM turned into N+M by have a narrow waist of access patterns, but the access patterns emerge from making data structures and algorithms orthogonal. IP orthogonalizes rendezvous and interaction. Text orthogonalizes transport and schema. The relational database orthogonalizes data query and manipulation and data storage and integrity. I think this distinction is important because Win32’s kernel object handles are a narrower waist than text, but what they orthogonalize (system resources and program access to them) is very different.
Orthogonalizing transport and schema leads me to another thing. One of the most important principles for writing a program that deals with external data is to parse at the boundaries, as early as possible. I don’t think we have spent effort on lightweight deserialization and verification. I’m just going to fanboy briefly about CUE here. Typed feature structures really do seem to be a big step forwards in describing the de facto languages we serialize and deserialize.
In that vein, I have more and more started to thing that unification is the right place for system manipulation stuff instead of the usual form of shell. When you get to sophisticated machine control, you switch to systems like ansible or chef, which are basically trying to do unification but poorly. Instead what if we set up a the idea of a description of state in Horn clauses or similar and then have the notion of querying with a clause or applying a clause. This would orthogonalize system description from application steps.
So maybe what we want is not a Bourne-like shell but a unification based system that makes it really easy to specify schema and desired state. The paper would have to be called “Unification: the ultimate shell” in homage to the lambda papers :)
Yes I agree about orthogonality –that’s another way of looking at the same principles. I’m not sure I agree about the symptom / cause framing – the goal is to design general, long-lasting systems with a minimum of code and testing. The narrow waist is an idea to help you with that goal, derived from existing systems that achieved it. But I do agree about orthogonality.
The Windows file handle case is very similar to file descriptors, and that was part of the discussion in the “Unix Philosophy” draft which I haven’t published.
The C++ STL example is also related, and gets at how to achieve SOME narrow waist properties in a statically typed context. I didn’t fully explain this comment at the end in the appendix:
e.g. every language is defined by its core data structures, and C++ has a harder problem because it needs to be efficient. And the tradeoff is a lot of compile time complexity, and more mechanisms like C++ 20 concepts.
It also relates to the protobuf code explosion example linked – you also have template code explosion. Not in source code, but in generated code. (In fact this is a major reason the oil-native binary is bigger than bash now – it uses C++ templates.)
I agree that we need parsing and serialization to be easier. Some people complain about text, but I think the only path is not to get rid of it, but to add features to languages that make dealing with it easier.
I did some experiments with Prolog and unification. From my perspective they are a useful model for some problems – maybe that set of problems is larger than I think. But I don’t think they fall in the same space shell.
Shell is more in the space of:
How do I massage some data from 3 web services and then turn that into Horn clauses, and then run unification on it?
How do I use unification and a SAT solver together in a pipeline? Or a Nvidia GPU?
It’s a lowest common denominator to bridge these more opinionated and locally more powerful models.
The Windows file handle case is very similar to file descriptors, and that was part of the discussion in the “Unix Philosophy” draft which I haven’t published.
The main difference between them is a great case study: POSIX file descriptors must be assigned as the lowest unused number. This happened to be the case on the original UNIX as a side effect of the implementation and so it was then used in the UNIX shell for redirection (close stdout, open file, close stdin, open file, and so on). It has not been necessary for anything since the dup2 system call was added. If *NIX didn’t try aggressively to restrict the number of system calls and system call arguments then you can imagine variants of open, socket and so on that all combined their functionality with dup2 and allocated a specific file descriptor number. The requirement to synchronise this allocation is a scalability bottleneck for modern *NIX systems - one of the reasons that io_uring gets a speedup is that it has a per-ring file-descriptor table.
Here, the narrow interface was slightly too narrow. If UNIX had started life with dup2 and not made this guarantee then the abstractions would be more orthogonal. The deterministic naming scheme also means that this is how you inherit file descriptors in child processes (which is very painful on operating systems that don’t implement closefrom). In contrast, in Windows the granting of access and then naming are separate. You can send handles to a child process (or any other process) but you then need some other mechanism to notify the process of the name that it should use. The loader will do this by writing a few handles into some global variables in a DLL’s memory, other things need different mechanisms. In UNIX, you need to send file descriptors via a UNIX domain socket, which means that you have a completely different mechanism for inheriting versus sending to a non-child process and it also means that you have a bootstrapping problem (how do you pass the file descriptor for the receive end of the socket to the other process?).
I think this is a great example of how an orthogonal interface can be built cleanly and ends up being more scalable and more flexible than the narrow interface.
But why do you think Unix is narrow in that sense?
I probably did not explain it very well, but the semantics without the lowest FD rule is narrower, because it has fewer requirements. Unix semantics are too wide, not too narrow. (It doesn’t help that there are two senses of the word narrow, mentioned in the post.)
I added a section to the post on “versionless evolution” related to this, citing Rich Hickey’s framing of “strengthen a promise” and “relax a requirement” as two directions of compatible evolution:
The problem is that a Unix implementation made an implicit promise a long time ago, and POSIX set that promise in stone. It can’t break the promise without breaking applications, so we’re stuck.
So the waist basically ONLY gets wider over time – that can be done by adding or strengthening promises, or relaxing requirements. You can’t remove any promises.
Any new feature adds a promise … Relaxing requirements is rarer, but does happen. In addition to my self-closing tag example on the web, I guess an example would be defining undefined behavior in C. That generally relaxes requirements on which combinations of language features are valid.
(Although compiled languages have a little more flexibility in evolution – they occasionally break things, as long as a reasonable error message is given and it doesn’t affect too many programs. Linux ABIs and the web get very messy because the composition is done at runtime and you don’t have the “safety check” of a compiler.)
I should also explore the relationship of this style of evolution to the covariance / contravariance issues on the “On
Variance and Extensibility” blog post linked, which is good.
But why do you think Unix is narrow in that sense?
Because it conflates a bunch of things into the FD abstraction. Most importantly, naming. The fact that FDs can be any integer and you have control over that value means that it’s easy to create a child process that has FDs in the right set of slots, without any other communication. In contrast, Windows HANDLEs are pointer values (or, at least, pointer-sized values) and if you send one to another process then you have no control over the value that it has in the target. This is a wider interface because it requires that you separate naming and authority in your tokens of authority but it then leads to more orthogonality because sending a HANDLE to a remote process is the same whether it’s a child process that you’re creating or one that you’ve somehow gained the ability to communicate with.
Imagine a UNIX without dup2 and where standard input, output, and error were not file descriptors 0, 1, and 2. Such a UNIX would probably need to use something like ELF auxiliary arguments to notify a new process of its initial file descriptor set and so its execve equivalent would also have been extended to provide a list of file descriptors and their names, which the startup code could parse and libc could then use to provide a getenv-like interface to read specific ones. STDIN_FILENO might be a macro that expanded to getfd("stdin"), or it might be that you’d just store that FD value in a global called STDIN_FILENO. Now your interface is wider because you need to have a separate naming mechanism, but it’s also more orthogonal and so the way that you look up a file descriptor with a well-known name that you inherit and the way that you’d look up one that was dynamically provided could be the same.
Yeah I’d say Windows is definitely wider than Unix. I was doing some research and I was curious if Windows had dup2(). And then I look at DuplicateDescriptor() and I’m like “why does it have so many arguments??” Generally speaking, more arguments and more functions is wider, although it’s certainly not a hard rule.
I was thinking of something more like this comparison:
Unix as is – with the rule that open() returns the lowest available FD
Unix that still has 0, 1, 2 open by default, but open() returns any descriptor number that’s not used
I would call the first one wider and the second one narrower. The second one makes fewer promises.
It might take more code to implement, but wide/narrow refers to the size of the interface, not the size of the code. And I suspect that’s why Unix is the way it is – because it was short/easy to implement! (like so many things in Unix)
I would liken it to a recent Python change to guarantee the order of dictionary iteration. It used to be that this was indeterminate.
Then they changed the hash table implementation, and it became determinate, so after some debate they made it part of the API.
This change can ONLY go one way – you can guarantee the order, but you can’t make it unspecified again once you’ve guaranteed it.
With file descriptors, if the integers are random, then you could change them to be contiguous, but not the reverse.
Thus the waist only gets wider over time. You can make a new promise (ordering, lowest FD), but you can’t remove a promise without breaking the waist.
So basically Unix is narrow compared to Windows but I’d argue wider than what it could be, in that respect.
Another example is the ordering of PIDs, and whether APIs use both PIDs and FDs (self pipe trick argument). Arguably it could be narrower.
Yeah I’d say Windows is definitely wider than Unix. I was doing some research and I was curious if Windows had dup2(). And then I look at DuplicateDescriptor() and I’m like “why does it have so many arguments??” Generally speaking, more arguments and more functions is wider, although it’s certainly not a hard rule.
DuplicateHandle is quite different from dup2 in a couple of ways. First, it does not give you any control over the value of the resulting HANDLE. This means that you can’t use it to replace an existing HANDLE, which is a big part of the separation of concerns: naming is not part of the HANDLE abstraction. Secondly, it combines the functionality of UNIX domain sockets out-of-band data for passing HANDLEs: if you have a HANDLE to the target process then you can inject a HANDLE into it. It also includes the equivalent of dup3’s close-on-exec flag.
I was thinking of something more like this comparison:
Unix as is – with the rule that open() returns the lowest available FD
Unix that still has 0, 1, 2 open by default, but open() returns any descriptor number that’s not used
I would call the first one wider and the second one narrower. The second one makes fewer promises.
I don’t think that you can look at these in isolation. The second one requires either the first or something like dup2, which doesn’t just duplicate a handle, it closes an existing one and (atomically) assigns the new one to the old existing’s name. This is narrow in the sense that it has very few parameters but it’s decidedly non-orthogonal because it conflates three different operations in a single call:
Duplicating a handle.
Assigning a specific name in a process-global namespace to that handle.
Closing an existing handle.
The original dup is somewhat better in that it conflates only the first and second of these. If you didn’t have these and wanted equivalent behavior then your userspace code would have to maintain a global data structure that every open call would update. If you tried to put that in a standard library for any language, people would laugh at you: why is your call that creates a handle also acquiring a lock and inserting it into a dictionary? That’s a completely orthogonal concern that could be built on top for anything that needs it.
When you consider the broader API surface, it’s really not clear to me which is wider, but the Win32 one is definitely more orthogonal.
Waist vs orthogonalization may be a difference in how we lay out systems. In my mind, I tend to constrain an architecture into existence, so orthogonalization feels like the natural concern. If you start with a concrete architecture that you transform to arrive where you want, then I could see waists being the natural concern.
My interest in unification for shells comes from the question of why we persist in using a teletype emulator. I think the answer is that we really, really need the accumulated state to make sense of what’s going on (which is why things like Jupyter Notebooks are a crappy environment for anything like systems administration), and the teletype emulator is really good at that. So if we want to have a shell-like UI for a system that isn’t a teletype emulator, we need something that doesn’t depend on accumulated state that way. Unification seemed like the most promising route for that.
Parsing is definitely undersupported. If you look at how bioinformaticists use shell, you’ll find that they do a huge amount of work (possibly the majority of their work) munging data into different formats between tools orchestrated by the shell. I got big wins for bioinformatics workflows off creating an opinionated layer for munging stuff into and out of Python.
What I envision more generally is being able to specify a structure in something like Cue, and then pipe text into that and have it either give me good errors or a parsed structure. Likewise, being able to pipe a structure into a specification of how it serializes.
Another example for your waist list: the model in MVC, where updates and notification are forced through a bottleneck.
Hm you might be right on average about state and shell / Jupyter notebooks. But the funny thing is that I use shell mostly in a batch fashion with this “task file” pattern, and try not to accumulate any shell state. There is still file system state, but that is managed across shell sessions.
I usually have a tmux session open with 10-20 different shells going concurrently. So they each have a clean state, with the exception of the occasional virtualenv.
There was a good talk “I don’t like jupyter notebooks” by a software engineer who didn’t like all the non-reproducible state that scientists put in notebooks. I tend to agree
The revelation I had is that notebooks encourage programming with mutable global variables, and I haven’t used that style in a very long time …
Some blog posts on the unification idea would be interesting. To me it still seems only locally powerful, because how does it help you with say parsing?
I think you can do parsing in Prolog – by turning a linear algorithm into one of unknown time complexity, which is bad engineering.
In my mind it’s better to just parse with Python or C or a parser generator, and then use that data to set up some kind of unification problem (again gluing it all together with a shell script :) ).
My experience is that the “glue” often dominates the actual logic of the system. Like if you’re doing AI, almost all the code is data preparation and automation and evaluation, and the actual algorithm itself is like 100 or 1000 lines of logic.
I looked at many of the scientific workflow systems, which seemed to be used in so many domains including bioinformatics, and are VERY shell-like. But I haven’t used any of them.
I haven’t used CUE but as far as I remember it’s derived from Google’s Borg config language. I used that a lot, and while most people seemed to dislike it, I thought it was fine. It has an interpreter that can be tested in the REPL, and that’s how I learned it, rather than trying to read docs :) It’s a pure functional language that evaluates quickly, which IMO is pretty simple to learn … I would like to see some elaboration on how CUE improves upon that.
Yeah MVC is an interesting decoupling. Although I think the waist idea already needs more refinement – I think I might put the static examples like LLVM and C++ STL in a different category. Most are dynamic and long lived – Internet and web scale.
Task files look like a neat idea. I’ll go read what you’ve linked to about it.
A lot of what I work on has a fairly complicated state that I then do things with, such as having a description of a managed Cassandra cluster and then deploying parts of it or querying parts of it. It’s less about building up state than having an implicit topic. The Ruby style blocks in Oil might be well suited for that. Kind of like having a hugely complicated version of CWD, but what is valid to do to that topic depends on what has been done already.
Actually, I could probably fix that pretty easily. Maybe I’ll do that today…
Some blog posts on the unification idea would be interesting. To me it still seems only locally powerful, because how does it help you with say parsing?
My notion is essentially using it to declare a grammar. If I have CSV coming in, I declare a “schema” that describes a data structure being embedded in the CSV, something like (this isn’t even vaguely valid)
which we try to unify with an actual incoming text and get back out a list of tuples. Maybe it’s the wrong approach entirely, but the phases of text <-> format <-> structure seem crucial and it would be nice to make the same declaration work in both directions and be able to compose them all in the same place. And maybe having that be a separate tool that we invoke and then get structured data back like you’re trying to do is the right way to go.
I actually think that the data preparation and cleaning parts of all data work could use a lot more tooling in this exact vein.
I looked at many of the scientific workflow systems, which seemed to be used in so many domains including bioinformatics, and are VERY shell-like. But I haven’t used any of them.
This is not accidental. Most of them come from people getting frustrated with shell and trying to build something a little better given the reality that tools in bioinformatics are published as Unix command line tools, often with their own input and output formats.
I haven’t used CUE but as far as I remember it’s derived from Google’s Borg config language…It’s a pure functional language that evaluates quickly, which IMO is pretty simple to learn …
CUE’s foundations are quite different. It’s graph unification over typed features structures. It’s what the author claims that he wanted to do for Borg but didn’t get to. :)
Text as a narrow waist is at odds with fine-grained, static types. My goal is to highlight tradeoffs, and analyze situations where each style is natural and efficient . . . Types are models; data from network/disk is reality. . . . The real issues [governing static vs. dynamic typing] are scale in space and time, heterogeneity, and extensibility!
Types are definitely models, usually defined and enforced within the authoritative domain of a specific programming language. But data (bytes) from network/disk is also a model, defined and enforced by an authority which is “lower” in the OSI Layers sense.
Static and dynamic are modifiers that constrain and disinhibit, respectively, concepts at a specific level of abstraction. So static or dynamic types at a programming language level, sure. But if you’re trying to build a model (or essay :wink:) that spans over more than one of these layers, that accommodates bytes-on-the-wire and types-in-a-language, then the concepts of static and dynamic aren’t super… useful?
Specifically, in your waist model, is there even a coherent concept something which can be static or dynamic? Or maybe I misunderstand the point here. I dunno. Anyway, interesting.
But data (bytes) from network/disk is also a model
Hm I think that is abusing the word “model”. What I mean is whether your program WORKS or not depends on the data, not the types. The types are generally gone at runtime. (I edited the post to say static types are models; data is reality.)
I’m using the words “model” and “map” in their dictionary senses – they’re useful representations or approximations of a thing in reality. But they are not the thing itself, which is what you want to control or reason about.
Data is what the user cares about, not types. The user cares if their pizza order was sent to the restaurant and they got back the mushroom and peppers they asked for. They don’t care what type system is used to express that!
When your model (encoding of a problem in a type system) matches reality, everything is great. You can reason about runtime behavior with the static types your programming language has. (And note that the singular “programming language” is a huge limitation! Most systems are written in more than one language, as I point out in the post.)
But there are many cases when the model doesn’t capture reality, and even disagrees with it. If you compile the wrong schema version into your binary, which I’ve seen happen. Or when you simply have a weak type system or inappropriate one (e.g. OCaml is a lot better for modeling languages than Pascal.)
e.g. the part about administrators / operators / SREs. Their entire job revolves around a dynamic view of software. If static type systems modeled everything about reality correctly, then they would have no job to do. (i.e. provisioning and load balancing are dynamic; security is dynamic, etc.)
Specifically, in your waist model, is there even a coherent concept something which can be static or dynamic?
I feel like I’m also using these words in a pretty conventional sense. The intro contrasts the two different styles associated with static and dynamic. I’m talking mainly about the dynamic side of things, and the “tradeoffs” section notes many limitations of a static view.
Static basically means “something that can be done without knowledge of the value of runtime data”. For example, the C++ or OCaml compiler has no idea what the value of argv is; it only knows its type.
The waists are almost all dynamic, although I point out the exception of LLVM.
I also note the difference between kernel APIs and ABIs with the Illumos Docker example.
When you upload a container to a cloud system, and it runs it, that’s dynamic runtime composition of your application and the remote kernel. You never compiled against the kernel headers on that machine, which could be (and are likely to be) different than the ones on your dev machine.
If you run ./configure; make, and then run the program on your machine, that’s more static because the compiler can give you some errors.
Most of the time things work when uploading containers, but if they don’t, the errors are pretty odd. It’s a little like running an x86 binary on an ARM system …
When your model (encoding of a problem in a type system) matches reality, everything is great.
“All models are wrong; some are useful.” —somebody
No model matches reality. That’s the point of models. They make simplifying assumptions about the thing they model, and encapsulate the complexity of that thing, so that subsequent layers can build upon that thing without needing to fully understand all of its particulars.
Data is what the user cares about, not types. The user cares if their pizza order was sent to the restaurant and they got back the mushroom and peppers they asked for. They don’t care what type system is used to express that!
“Data” isn’t well defined, it’s a model of something else. An order for a pizza with mushrooms is a domain concept at the Dominoes-to-customer layer of abstraction. You can call that data, sure. But that order gets recorded as a row in a database, and that row is also data. A JSON representation of that row is data. The bytes that the JSON object marshals to is data. A byte is a model of a set of bits. A bit is a model of one of two possibilities rendered in some physical medium — a signal on a wire, a location on a magnetic tape, etc. 5 volts for 1 nanosecond on a bit of copper models something else. And so on.
When you upload a container to a cloud system, and it runs it, that’s dynamic runtime composition of your application and the remote kernel. You never compiled against the kernel headers on that machine, which could be (and are likely to be) different than the ones on your dev machine.
If you run ./configure; make, and then run the program on your machine, that’s more static because the compiler can give you some errors.
It seems like you’re describing a model :wink: where there is some concrete and immutable artifact, which is executed or applied in some environment. And you’re defining static/dynamic as like how much information is captured by that immutable artifact, vs. how much information is left to the runtime environment? This isn’t wrong of course but I don’t think it’s like objectively correct either. I don’t think there is a correct answer! Static/dynamic are modifiers for concepts within a domain, not well-defined concepts themselves.
Yeah I take your point that data can also “model” a real-world domain (in addition to static types modeling data). I shouldn’t have used the pizza example, because that’s a case where you can imagine more space between the two.
I think the better example is the container one – I would rephrase it as: The user doesn’t care what static types are on the remote system or even the local system. They care if their container actually runs correctly on the remote machine. The data has to be compatible: the ABI that executable code uses has to be compatible, as well as service interfaces that the container requires.
My view is that for this problem, “static types” – the models – are quite secondary and do very little to help you with the problem. The models don’t capture reality well (the ABI and service interfaces).
I agree that static/dynamic are very broad terms that have slightly different meanings depending on the context, but I think it is clear in the 4 motivating design questions I gave:
Shell. Shell is extremely dynamic, one of the most dynamic languages. It invokes external processes; there is no common type system.
JSON. Also very dynamic. There is a very strong analogy:
Protobuf : Static languages like C++ :: JSON : Dynamic languages like Python/JS
i.e. protobuf is a static version of JSON, or JSON is a dynamic version of protobuf
(replace protobuf with Thrift or maybe even Corba)
Kubernetes. Because it’s a big polyglot distributed system, all the tradeoffs / downsides / inapplicability of the static composition style apply directly to this problem.
Docker. Docker is both a build tool on the dev machine and a runtime on a cluster. It has both concerns, but mostly dynamic ones in my view.
I hope that makes it clearer … If there are some alternative terms you suggest, I’m open to it.
If what you’re saying is that the “narrow waist” itself is a model… Well sure, but the question is whether it is useful or not. I think it’s specific enough not to be obvious or trivial, but not so precise that it’s brittle and can’t admit variation.
What I said in a sibling comment:
the goal is to design general, long-lasting systems with a minimum of code and testing. The narrow waist is an idea to help you with that goal, derived from existing systems that achieved it.
The user doesn’t care what static types are on the remote system or even the local system. They care if their container actually runs correctly on the remote machine. The data has to be compatible: the ABI that executable code uses has to be compatible, as well as service interfaces that the container requires.
My view is that for this problem, “static types” – the models – are quite secondary and do very little to help you with the problem. The models don’t capture reality well (the ABI and service interfaces).
“Reality” isn’t the ABI and service interfaces, though. Those are also models! And “static types” are models, but so are “dynamic” types. Nothing is reality! Everything is a model. “Reality” is an arbitrary label applied to a specific layer of abstraction which happens to match the user’s expectations. It’s not objective. “Data” is not the truth.
A way that I’ve been thinking about this is “early binding vs late binding”. The Alan Kay quote is “OOP to me means only messaging, local retention, and protection and hiding of state-process, and extreme late-binding of all things.” So he’s very pro-late binding. But early binding (can) produce more efficient and robust systems, at least theoretically. Going through before hand, making sure all bindings are correct and then optimizing them should give you a faster, more reliable system. At the very least, it can’t hurt. On the other hand, the reason Kay likes late binding is that the system is better for building new things if all the parts are bound at the last moment.
This applies fractally:
Dynamic sacrifices performance and reliability in exchange for ease of continued development (source is available on disk; types are loose). In cases where performance is well passed good enough, this seems like a good tradeoff.
I think what the “narrow waist” concept adds is a description of when the advantages of the dynamic system are likely to compound rather than the liabilities. VBScript was dynamic, but it quickly became an unmaintainable mess. Having a big commercial website that links together 6 separate PHP and Java backends via an Enterprise Service Bus is going to bad ugly and slow, but it will work because the ESB is a narrow waist.
Yeah that’s is fair. I don’t use the term that much (maybe because almost everything I do is late bound), but wikipedia says it’s synonymous with “dynamic binding”:
https://en.wikipedia.org/wiki/Late_binding
I just call that “dynamic”, so it probably depends on your personal history of problems and domains. I’m more focused on systems, whereas others might be more focused on GUIs as an application of OO.
Well I’d say the narrow waist is first about interoperability and economy of code. And that does seem to come with a lot of dynamic mechanisms, as with shell, Unix, the web, and distributed systems.
So yeah the post mixes many topics. It started out with “narrow waists”, but then I could not help addressing the dynamic vs. static styles. I added the #1 and #2 dichotomy in the intro toward the very end. It was something of a surprising relationship but it does appear to hold up after looking at all the examples, the history of the biggest evolved systems in software.
There are some examples of static narrow waists – as mentioned LLVM, and STL design. Not sure what to do with those – maybe I should “define them out” of what I’m making claims about.
I feel like the narrow waist is a symptom not a cause. The cause is trying for orthogonality (hearkening back to the goals of ALGOL 68). For example, in the STL we explicitly get NxM turned into N+M by have a narrow waist of access patterns, but the access patterns emerge from making data structures and algorithms orthogonal. IP orthogonalizes rendezvous and interaction. Text orthogonalizes transport and schema. The relational database orthogonalizes data query and manipulation and data storage and integrity. I think this distinction is important because Win32’s kernel object handles are a narrower waist than text, but what they orthogonalize (system resources and program access to them) is very different.
Orthogonalizing transport and schema leads me to another thing. One of the most important principles for writing a program that deals with external data is to parse at the boundaries, as early as possible. I don’t think we have spent effort on lightweight deserialization and verification. I’m just going to fanboy briefly about CUE here. Typed feature structures really do seem to be a big step forwards in describing the de facto languages we serialize and deserialize.
In that vein, I have more and more started to thing that unification is the right place for system manipulation stuff instead of the usual form of shell. When you get to sophisticated machine control, you switch to systems like ansible or chef, which are basically trying to do unification but poorly. Instead what if we set up a the idea of a description of state in Horn clauses or similar and then have the notion of querying with a clause or applying a clause. This would orthogonalize system description from application steps.
So maybe what we want is not a Bourne-like shell but a unification based system that makes it really easy to specify schema and desired state. The paper would have to be called “Unification: the ultimate shell” in homage to the lambda papers :)
Yes I agree about orthogonality –that’s another way of looking at the same principles. I’m not sure I agree about the symptom / cause framing – the goal is to design general, long-lasting systems with a minimum of code and testing. The narrow waist is an idea to help you with that goal, derived from existing systems that achieved it. But I do agree about orthogonality.
The Windows file handle case is very similar to file descriptors, and that was part of the discussion in the “Unix Philosophy” draft which I haven’t published.
The C++ STL example is also related, and gets at how to achieve SOME narrow waist properties in a statically typed context. I didn’t fully explain this comment at the end in the appendix:
https://old.reddit.com/r/ProgrammingLanguages/comments/lliyuo/are_there_any_interesting_programming_languages/gnpx0so/
e.g. every language is defined by its core data structures, and C++ has a harder problem because it needs to be efficient. And the tradeoff is a lot of compile time complexity, and more mechanisms like C++ 20 concepts.
It also relates to the protobuf code explosion example linked – you also have template code explosion. Not in source code, but in generated code. (In fact this is a major reason the oil-native binary is bigger than bash now – it uses C++ templates.)
I agree that we need parsing and serialization to be easier. Some people complain about text, but I think the only path is not to get rid of it, but to add features to languages that make dealing with it easier.
I did some experiments with Prolog and unification. From my perspective they are a useful model for some problems – maybe that set of problems is larger than I think. But I don’t think they fall in the same space shell.
Shell is more in the space of:
It’s a lowest common denominator to bridge these more opinionated and locally more powerful models.
This post has a bunch of slogans around that: http://www.oilshell.org/blog/2021/01/philosophy-design.html
The main difference between them is a great case study: POSIX file descriptors must be assigned as the lowest unused number. This happened to be the case on the original UNIX as a side effect of the implementation and so it was then used in the UNIX shell for redirection (close stdout, open file, close stdin, open file, and so on). It has not been necessary for anything since the
dup2system call was added. If *NIX didn’t try aggressively to restrict the number of system calls and system call arguments then you can imagine variants ofopen,socketand so on that all combined their functionality withdup2and allocated a specific file descriptor number. The requirement to synchronise this allocation is a scalability bottleneck for modern *NIX systems - one of the reasons thatio_uringgets a speedup is that it has a per-ring file-descriptor table.Here, the narrow interface was slightly too narrow. If UNIX had started life with
dup2and not made this guarantee then the abstractions would be more orthogonal. The deterministic naming scheme also means that this is how you inherit file descriptors in child processes (which is very painful on operating systems that don’t implementclosefrom). In contrast, in Windows the granting of access and then naming are separate. You can send handles to a child process (or any other process) but you then need some other mechanism to notify the process of the name that it should use. The loader will do this by writing a few handles into some global variables in a DLL’s memory, other things need different mechanisms. In UNIX, you need to send file descriptors via a UNIX domain socket, which means that you have a completely different mechanism for inheriting versus sending to a non-child process and it also means that you have a bootstrapping problem (how do you pass the file descriptor for the receive end of the socket to the other process?).I think this is a great example of how an orthogonal interface can be built cleanly and ends up being more scalable and more flexible than the narrow interface.
Yes, nice example. I remember that from the Scalable Commutativity Rule paper several years ago:
https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf
(I probably should re-read this paper)
But why do you think Unix is narrow in that sense?
I probably did not explain it very well, but the semantics without the lowest FD rule is narrower, because it has fewer requirements. Unix semantics are too wide, not too narrow. (It doesn’t help that there are two senses of the word narrow, mentioned in the post.)
I added a section to the post on “versionless evolution” related to this, citing Rich Hickey’s framing of “strengthen a promise” and “relax a requirement” as two directions of compatible evolution:
http://www.oilshell.org/blog/2022/03/backlog-arch.html#the-web-evolved-in-a-versionless-manner
The problem is that a Unix implementation made an implicit promise a long time ago, and POSIX set that promise in stone. It can’t break the promise without breaking applications, so we’re stuck.
So the waist basically ONLY gets wider over time – that can be done by adding or strengthening promises, or relaxing requirements. You can’t remove any promises.
Any new feature adds a promise … Relaxing requirements is rarer, but does happen. In addition to my self-closing tag example on the web, I guess an example would be defining undefined behavior in C. That generally relaxes requirements on which combinations of language features are valid.
(Although compiled languages have a little more flexibility in evolution – they occasionally break things, as long as a reasonable error message is given and it doesn’t affect too many programs. Linux ABIs and the web get very messy because the composition is done at runtime and you don’t have the “safety check” of a compiler.)
I should also explore the relationship of this style of evolution to the covariance / contravariance issues on the “On Variance and Extensibility” blog post linked, which is good.
Because it conflates a bunch of things into the FD abstraction. Most importantly, naming. The fact that FDs can be any integer and you have control over that value means that it’s easy to create a child process that has FDs in the right set of slots, without any other communication. In contrast, Windows
HANDLEs are pointer values (or, at least, pointer-sized values) and if you send one to another process then you have no control over the value that it has in the target. This is a wider interface because it requires that you separate naming and authority in your tokens of authority but it then leads to more orthogonality because sending aHANDLEto a remote process is the same whether it’s a child process that you’re creating or one that you’ve somehow gained the ability to communicate with.Imagine a UNIX without
dup2and where standard input, output, and error were not file descriptors 0, 1, and 2. Such a UNIX would probably need to use something like ELF auxiliary arguments to notify a new process of its initial file descriptor set and so itsexecveequivalent would also have been extended to provide a list of file descriptors and their names, which the startup code could parse and libc could then use to provide agetenv-like interface to read specific ones.STDIN_FILENOmight be a macro that expanded togetfd("stdin"), or it might be that you’d just store that FD value in a global calledSTDIN_FILENO. Now your interface is wider because you need to have a separate naming mechanism, but it’s also more orthogonal and so the way that you look up a file descriptor with a well-known name that you inherit and the way that you’d look up one that was dynamically provided could be the same.Yeah I’d say Windows is definitely wider than Unix. I was doing some research and I was curious if Windows had dup2(). And then I look at DuplicateDescriptor() and I’m like “why does it have so many arguments??” Generally speaking, more arguments and more functions is wider, although it’s certainly not a hard rule.
I was thinking of something more like this comparison:
I would call the first one wider and the second one narrower. The second one makes fewer promises.
It might take more code to implement, but wide/narrow refers to the size of the interface, not the size of the code. And I suspect that’s why Unix is the way it is – because it was short/easy to implement! (like so many things in Unix)
I would liken it to a recent Python change to guarantee the order of dictionary iteration. It used to be that this was indeterminate.
Then they changed the hash table implementation, and it became determinate, so after some debate they made it part of the API.
This change can ONLY go one way – you can guarantee the order, but you can’t make it unspecified again once you’ve guaranteed it.
With file descriptors, if the integers are random, then you could change them to be contiguous, but not the reverse.
Thus the waist only gets wider over time. You can make a new promise (ordering, lowest FD), but you can’t remove a promise without breaking the waist.
So basically Unix is narrow compared to Windows but I’d argue wider than what it could be, in that respect.
Another example is the ordering of PIDs, and whether APIs use both PIDs and FDs (self pipe trick argument). Arguably it could be narrower.
DuplicateHandleis quite different fromdup2in a couple of ways. First, it does not give you any control over the value of the resultingHANDLE. This means that you can’t use it to replace an existingHANDLE, which is a big part of the separation of concerns: naming is not part of theHANDLEabstraction. Secondly, it combines the functionality of UNIX domain sockets out-of-band data for passingHANDLEs: if you have aHANDLEto the target process then you can inject aHANDLEinto it. It also includes the equivalent ofdup3’s close-on-exec flag.I don’t think that you can look at these in isolation. The second one requires either the first or something like
dup2, which doesn’t just duplicate a handle, it closes an existing one and (atomically) assigns the new one to the old existing’s name. This is narrow in the sense that it has very few parameters but it’s decidedly non-orthogonal because it conflates three different operations in a single call:The original
dupis somewhat better in that it conflates only the first and second of these. If you didn’t have these and wanted equivalent behavior then your userspace code would have to maintain a global data structure that everyopencall would update. If you tried to put that in a standard library for any language, people would laugh at you: why is your call that creates a handle also acquiring a lock and inserting it into a dictionary? That’s a completely orthogonal concern that could be built on top for anything that needs it.When you consider the broader API surface, it’s really not clear to me which is wider, but the Win32 one is definitely more orthogonal.
Waist vs orthogonalization may be a difference in how we lay out systems. In my mind, I tend to constrain an architecture into existence, so orthogonalization feels like the natural concern. If you start with a concrete architecture that you transform to arrive where you want, then I could see waists being the natural concern.
My interest in unification for shells comes from the question of why we persist in using a teletype emulator. I think the answer is that we really, really need the accumulated state to make sense of what’s going on (which is why things like Jupyter Notebooks are a crappy environment for anything like systems administration), and the teletype emulator is really good at that. So if we want to have a shell-like UI for a system that isn’t a teletype emulator, we need something that doesn’t depend on accumulated state that way. Unification seemed like the most promising route for that.
Parsing is definitely undersupported. If you look at how bioinformaticists use shell, you’ll find that they do a huge amount of work (possibly the majority of their work) munging data into different formats between tools orchestrated by the shell. I got big wins for bioinformatics workflows off creating an opinionated layer for munging stuff into and out of Python.
What I envision more generally is being able to specify a structure in something like Cue, and then pipe text into that and have it either give me good errors or a parsed structure. Likewise, being able to pipe a structure into a specification of how it serializes.
Another example for your waist list: the model in MVC, where updates and notification are forced through a bottleneck.
Hm you might be right on average about state and shell / Jupyter notebooks. But the funny thing is that I use shell mostly in a batch fashion with this “task file” pattern, and try not to accumulate any shell state. There is still file system state, but that is managed across shell sessions.
I usually have a tmux session open with 10-20 different shells going concurrently. So they each have a clean state, with the exception of the occasional virtualenv.
I describe the “task file” thing here a bit, but it’s a sketch: http://www.oilshell.org/blog/2020/02/good-parts-sketch.html#semi-automation-with-runsh-scripts
I might be in the monitory, but I’m not alone. I posted this article about the same shell usage pattern: https://lobste.rs/s/lob0rw/replacing_make_with_shell_script_for
There was a good talk “I don’t like jupyter notebooks” by a software engineer who didn’t like all the non-reproducible state that scientists put in notebooks. I tend to agree
https://www.youtube.com/watch?v=7jiPeIFXb6U
The revelation I had is that notebooks encourage programming with mutable global variables, and I haven’t used that style in a very long time …
Some blog posts on the unification idea would be interesting. To me it still seems only locally powerful, because how does it help you with say parsing?
I think you can do parsing in Prolog – by turning a linear algorithm into one of unknown time complexity, which is bad engineering.
In my mind it’s better to just parse with Python or C or a parser generator, and then use that data to set up some kind of unification problem (again gluing it all together with a shell script :) ).
My experience is that the “glue” often dominates the actual logic of the system. Like if you’re doing AI, almost all the code is data preparation and automation and evaluation, and the actual algorithm itself is like 100 or 1000 lines of logic.
I looked at many of the scientific workflow systems, which seemed to be used in so many domains including bioinformatics, and are VERY shell-like. But I haven’t used any of them.
I haven’t used CUE but as far as I remember it’s derived from Google’s Borg config language. I used that a lot, and while most people seemed to dislike it, I thought it was fine. It has an interpreter that can be tested in the REPL, and that’s how I learned it, rather than trying to read docs :) It’s a pure functional language that evaluates quickly, which IMO is pretty simple to learn … I would like to see some elaboration on how CUE improves upon that.
Yeah MVC is an interesting decoupling. Although I think the waist idea already needs more refinement – I think I might put the static examples like LLVM and C++ STL in a different category. Most are dynamic and long lived – Internet and web scale.
Same here, I don’t think it’s as uncommon as you seems to think.
Task files look like a neat idea. I’ll go read what you’ve linked to about it.
A lot of what I work on has a fairly complicated state that I then do things with, such as having a description of a managed Cassandra cluster and then deploying parts of it or querying parts of it. It’s less about building up state than having an implicit topic. The Ruby style blocks in Oil might be well suited for that. Kind of like having a hugely complicated version of CWD, but what is valid to do to that topic depends on what has been done already.
Actually, I could probably fix that pretty easily. Maybe I’ll do that today…
My notion is essentially using it to declare a grammar. If I have CSV coming in, I declare a “schema” that describes a data structure being embedded in the CSV, something like (this isn’t even vaguely valid)
lines(Input, Lines), Lines = [csv_line(Name,Age,Address)] |- [(Name,Age,Address)]
which we try to unify with an actual incoming text and get back out a list of tuples. Maybe it’s the wrong approach entirely, but the phases of text <-> format <-> structure seem crucial and it would be nice to make the same declaration work in both directions and be able to compose them all in the same place. And maybe having that be a separate tool that we invoke and then get structured data back like you’re trying to do is the right way to go.
I actually think that the data preparation and cleaning parts of all data work could use a lot more tooling in this exact vein.
This is not accidental. Most of them come from people getting frustrated with shell and trying to build something a little better given the reality that tools in bioinformatics are published as Unix command line tools, often with their own input and output formats.
CUE’s foundations are quite different. It’s graph unification over typed features structures. It’s what the author claims that he wanted to do for Borg but didn’t get to. :)
Types are definitely models, usually defined and enforced within the authoritative domain of a specific programming language. But data (bytes) from network/disk is also a model, defined and enforced by an authority which is “lower” in the OSI Layers sense.
Static and dynamic are modifiers that constrain and disinhibit, respectively, concepts at a specific level of abstraction. So static or dynamic types at a programming language level, sure. But if you’re trying to build a model (or essay :wink:) that spans over more than one of these layers, that accommodates bytes-on-the-wire and types-in-a-language, then the concepts of static and dynamic aren’t super… useful?
Specifically, in your waist model, is there even a coherent concept something which can be static or dynamic? Or maybe I misunderstand the point here. I dunno. Anyway, interesting.
Hm I think that is abusing the word “model”. What I mean is whether your program WORKS or not depends on the data, not the types. The types are generally gone at runtime. (I edited the post to say static types are models; data is reality.)
I’m using the words “model” and “map” in their dictionary senses – they’re useful representations or approximations of a thing in reality. But they are not the thing itself, which is what you want to control or reason about.
Data is what the user cares about, not types. The user cares if their pizza order was sent to the restaurant and they got back the mushroom and peppers they asked for. They don’t care what type system is used to express that!
When your model (encoding of a problem in a type system) matches reality, everything is great. You can reason about runtime behavior with the static types your programming language has. (And note that the singular “programming language” is a huge limitation! Most systems are written in more than one language, as I point out in the post.)
But there are many cases when the model doesn’t capture reality, and even disagrees with it. If you compile the wrong schema version into your binary, which I’ve seen happen. Or when you simply have a weak type system or inappropriate one (e.g. OCaml is a lot better for modeling languages than Pascal.)
The fallacies shed some light on that:
https://www.oilshell.org/blog/2021/07/blog-backlog-1.html#fallacies
e.g. the part about administrators / operators / SREs. Their entire job revolves around a dynamic view of software. If static type systems modeled everything about reality correctly, then they would have no job to do. (i.e. provisioning and load balancing are dynamic; security is dynamic, etc.)
I feel like I’m also using these words in a pretty conventional sense. The intro contrasts the two different styles associated with static and dynamic. I’m talking mainly about the dynamic side of things, and the “tradeoffs” section notes many limitations of a static view.
Static basically means “something that can be done without knowledge of the value of runtime data”. For example, the C++ or OCaml compiler has no idea what the value of
argvis; it only knows its type.The waists are almost all dynamic, although I point out the exception of LLVM.
I also note the difference between kernel APIs and ABIs with the Illumos Docker example.
When you upload a container to a cloud system, and it runs it, that’s dynamic runtime composition of your application and the remote kernel. You never compiled against the kernel headers on that machine, which could be (and are likely to be) different than the ones on your dev machine.
If you run
./configure; make, and then run the program on your machine, that’s more static because the compiler can give you some errors.Most of the time things work when uploading containers, but if they don’t, the errors are pretty odd. It’s a little like running an x86 binary on an ARM system …
No model matches reality. That’s the point of models. They make simplifying assumptions about the thing they model, and encapsulate the complexity of that thing, so that subsequent layers can build upon that thing without needing to fully understand all of its particulars.
“Data” isn’t well defined, it’s a model of something else. An order for a pizza with mushrooms is a domain concept at the Dominoes-to-customer layer of abstraction. You can call that data, sure. But that order gets recorded as a row in a database, and that row is also data. A JSON representation of that row is data. The bytes that the JSON object marshals to is data. A byte is a model of a set of bits. A bit is a model of one of two possibilities rendered in some physical medium — a signal on a wire, a location on a magnetic tape, etc. 5 volts for 1 nanosecond on a bit of copper models something else. And so on.
It seems like you’re describing a model :wink: where there is some concrete and immutable artifact, which is executed or applied in some environment. And you’re defining static/dynamic as like how much information is captured by that immutable artifact, vs. how much information is left to the runtime environment? This isn’t wrong of course but I don’t think it’s like objectively correct either. I don’t think there is a correct answer! Static/dynamic are modifiers for concepts within a domain, not well-defined concepts themselves.
Yeah I take your point that data can also “model” a real-world domain (in addition to static types modeling data). I shouldn’t have used the pizza example, because that’s a case where you can imagine more space between the two.
I think the better example is the container one – I would rephrase it as: The user doesn’t care what static types are on the remote system or even the local system. They care if their container actually runs correctly on the remote machine. The data has to be compatible: the ABI that executable code uses has to be compatible, as well as service interfaces that the container requires.
My view is that for this problem, “static types” – the models – are quite secondary and do very little to help you with the problem. The models don’t capture reality well (the ABI and service interfaces).
I agree that static/dynamic are very broad terms that have slightly different meanings depending on the context, but I think it is clear in the 4 motivating design questions I gave:
Shell. Shell is extremely dynamic, one of the most dynamic languages. It invokes external processes; there is no common type system.
JSON. Also very dynamic. There is a very strong analogy:
i.e. protobuf is a static version of JSON, or JSON is a dynamic version of protobuf
(replace protobuf with Thrift or maybe even Corba)
Kubernetes. Because it’s a big polyglot distributed system, all the tradeoffs / downsides / inapplicability of the static composition style apply directly to this problem.
Docker. Docker is both a build tool on the dev machine and a runtime on a cluster. It has both concerns, but mostly dynamic ones in my view.
I hope that makes it clearer … If there are some alternative terms you suggest, I’m open to it.
If what you’re saying is that the “narrow waist” itself is a model… Well sure, but the question is whether it is useful or not. I think it’s specific enough not to be obvious or trivial, but not so precise that it’s brittle and can’t admit variation.
What I said in a sibling comment:
“Reality” isn’t the ABI and service interfaces, though. Those are also models! And “static types” are models, but so are “dynamic” types. Nothing is reality! Everything is a model. “Reality” is an arbitrary label applied to a specific layer of abstraction which happens to match the user’s expectations. It’s not objective. “Data” is not the truth.