1. 14

    Smalltalk-80 had orthogonal persistence. When you quit, it wrote your whole workspace (every object) to disk, and on restart it read it back in again. This was quite slow. Obviously reading/writing 32767 objects would be fast nowadays, but today’s apps and environments also use many orders of magnitude more data.

    The workspace was also kind of a pain for end users. It turns out documents are a really useful abstraction, but ST80 had no such concept. Any application software that wanted to allow users to interchange documents had to implement its own code for reading and writing files. So now you’re back to square one, just like a typical file-based environment.

    Another “fun” aspect of orthogonal persistence is that corrupted in-memory state is now persistent. If things go wonky due to a bug, and now let’s say typing anything causes a crash … but the underlying damage happened before your last workspace save … you’re hosed. You can’t just quit and restart, because you get the exact same corrupted state. Again, the only way to recover is to write your data to a separate file [without any typing!], restart a fresh blank VM, and read in the file.

    1. 6

      Thanks for sharing this history! Interesting how ideas in software always tend to repeat themselves.

      Another “fun” aspect of orthogonal persistence is that corrupted in-memory state is now persistent. If things go wonky due to a bug, and now let’s say typing anything causes a crash … but the underlying damage happened before your last workspace save … you’re hosed. You can’t just quit and restart, because you get the exact same corrupted state. Again, the only way to recover is to write your data to a separate file [without any typing!], restart a fresh blank VM, and read in the file.

      This seems like an obvious issue in retrospect. I wonder if this can be partially solved (or at least alleviated) by keeping a historical log of states. I feel like this idea of orthogonal persistence is somewhat similar to something like Elm, and with a time-traveling debugger (like Elm has) which persists the entire history of state, debugging things becomes really easy.

      1. 1

        I am also reminded of this tweet about EventSourcing. https://twitter.com/jbogard/status/1220378776753885185

        1. 1

          Hah! Isn’t that just true of basically every magic bullet solution though? Jokes aside, I can’t say how well this concept actually scales in Elm, as I only briefly worked with it, but the timetravel debugger was pretty cool and powerful.

      2. 3

        The corruption point is a good one! I hadn’t thought about it, but it does seem obvious in hindsight. I think one way to fix that is to log all the input to the module so it can be rebuilt at any point in the future, sort of like how urbit does it.

        1. 5

          I think a better approach to persistence is to use a database that holds data in a form fairly close to the app’s model objects (e.g. an object or document database.) That makes it pretty easy to add persistence to the model layer. Plus you get ACID. There are a number of apps that use SQLite databases as all or part of their persistent state, although IMO a relational db isn’t ideal for this.

          There’s not much point to persisting the views, it’s just wasted space. And the controller layer usually has only a bit of state to persist, like scroll positions or display modes.

          1. 3

            The object database point is exceptional. What ever happened to those ideas?

            1. 1

              Yeah, I don’t understand why object dbs didn’t get more popular. I’ve never used one, but they seem like a great fit for app persistence. Maybe it’s because Big Data was driving the adoption of non-relational dbs, making scalability the biggest criterion; my naive understanding is that object databases don’t scale well.

              All this is reminding me of the NewtonOS “Soup”; there were some good articles about it posted here a few months ago.

              1. 2

                As someone who is somewhat ignorant:

                How do object databases handle schema changes as the shape of data evolves?

                Can you do an ad-hoc join on not-formally-related data without dragging the whole dataset across the network ?

                1. 1

                  I don’t know; my knowledge is pretty superficial. Time to learn more.

                2. 2

                  Yeah, I don’t understand why object dbs didn’t get more popular. I’ve never used one, but they seem like a great fit for app persistence.

                  Agreed! Would love to explore more.

                  Maybe it’s because Big Data was driving the adoption of non-relational dbs, making scalability the biggest criterion; my naive understanding is that object databases don’t scale well.

                  Fascinating observation.

            2. 4

              When “all input” includes all user interaction, all network/IPC communication, and potentially large media/data… that gets tricky.

              There’s also, naturally, potential input which you don’t want recorded.

          1. 5

            Redox is not a particularly good example of operating system design

            A bold claim that I’d be curious to hear sustantiated.

            1. 4

              Someone on the reddit post asked much the same thing. Here’s my reply:

              Sure thing. I’m not saying that Redox isn’t a great thing. It was the first project to seriously open up the possibility of writing operating systems in Rust and I will forever be thankful to it for that.

              To elaborate on bad design: there are a number of questionable (at least to me) design decisions. For example, why are schemes designed the way they are? It seems to me like if they want to change the api between usermode and the kernel, they should either go all the way or none of it. Schemes change it a bit, but they don’t reconsider and fix bad design choices in posix and linux.

              Furthermore, Redox is simply not designed for performance. Even the scheduler, one of the most important contributors to the overall performance of a system does far more work than necessary. It exchanges performance for slightly more simplicity, which, when designing an operating system, is rarely the correct choice.

              1. 1

                I don’t understand how it’s possible pick three here: “full-native speed”, single address space OS (everything in ring 0) and security. I believe you can only pick two.

                1. 1

                  Well, that’s what nebulet is trying to challenge.

                    1. 1

                      I haven’t yet read the whole paper but in the conclusion they say that performance was a non-goal. They “also improved message-passing performance by enabling zero-copy communication through pointer passing”. Although I don’t see why zero-copy IPC can’t be implemented in a more traditional OS design.

                      The only (performance-related) advantage such design has in my opinion is cheaper context-switching, but I’m not convinced it’s worth it. Time (and benchmarks) will show, I guess.

                      1. 1

                        When communication across processes becomes cheaper than posting a message to a queue belonging to another thread in the same process in a more traditional design, I’d say that that’s quite a monstrous “only” benefit.

                        I should have drawn your attention to section 2.1 in the original comment, that’s where you original query is addressed. Basically the protection comes from static analysis, a bit like the original Native Client or Java’s bytecode verifier