1. 30
  1. 7

    The first thing that comes to my mind is to use this as the substrate for a securely sandboxed, distributed social-software platform, i.e. something like Urbit without the Nazi problem and baked-in feudalism. (And, for that matter, the willfully impenetrable custom programming language.)

    1. 2

      Web Assembly has various security issues such as the lack of read-only memory that I don’t see being fixed anytime in the near future: https://www.usenix.org/system/files/sec20-lehmann.pdf .

      1. 1

        Interesting. But those are vulnerabilities within a WASM process, which would not be as much of a problem for a compartmentalized design like Lunatic.

        1. 1

          That doesn’t make any sense. Of course it’s an issue and it’s not compartmentalized if the compartmentalization is itself broken because of this problem. The sand castle works until the tide washes it away.

          1. 1

            I’m assuming the issues described in that paper would allow exploits of buggy WASM code, that could allow an attacker to run their own WASM code in the process. But that they wouldn’t let the attacker break out of the WASM sandbox itself (because that would be a huge problem that would have to be fixed.)

            So: worst case is that one process within a Lunatic system is compromised. But the worst it can do is read incoming messages and send incorrect outgoing ones. Which might be a problem but not nearly as bad as in a non-compatmentalized design.

            1. 1

              No that’s incorrect. It completely allows breakout of the “sandbox”.

              How you are sandboxing that “one process”? What security controls are you relying on? Please describe in detail. You are going to need to provide some evidence on what exactly WASM is providing here. I think that linked paper is extremely clear on what is missing.

              Normal linux programs have a host of security controls like ASLR and read-only memory that limit what an attacker can do with vulnerable memory. If those controls are missing like they are in WASM than everything is hosed.

              WASM is a regressive security nightmare and is fundamentally broken.

    2. 3

      So this sounds pretty good. Are there detailed code examples of using code intended for shared nothing, and “normal” code? Basically the blurb sounds great but I want to see some of the substance.

      1. 8

        Hi! I’m the author of Lunatic. I have built a demo telnet chat application with it in Rust and also written a blog post about the structure of the app. I hope this helps you to get a feeling for the code structure.

        There is no separation between shared nothing and “normal” code. When you spawn a new process, the closure you pass to the spawn function will run in its own process. Everything you do inside this closure can’t access any memory from the parent process. That’s why the spawn function takes an initial context that will be copied into the child’s memory space. From the developers perspective it should be just writing “regular” Rust code.

        This is a Rust specific example, but for different languages this interaction is going to look differently. I always try to make it feel native to the language and libraries. Lunatic should feel like “just a library” for the developer.

        1. 10

          Can you really achieve “the massive concurrency of Erlang” using web assembly? The Erlang virtual machine and runtime are tuned for this use case (millions of communicating processes) and web assembly is not. Erlang processes have a very small memory footprint, with an initial default heap size of 233 words; the heap grows and shrinks dynamically as needed. A quick peek at Lunatic suggests a process memory is sized as a multiple of 64K blocks. There are other issues, like the speed of IPC. If you want to make this claim, it would be interesting to show comparative benchmarks of massive concurrency in Erlang vs Lunatic.

          1. 8

            The wasm spec itself requires that all memory used by a wasm process is a multiple of the wasm page size, which is 64Ki. So there’s probably nothing the author can do about that. At least nothing simple.

            But you raise a valid point. If every wasm process uses its minimum amount of memory, then you can only spawn ~15k processes per GB of host memory (I’m sure I introduced error somewhere due to GB/GiB).

            This is probably plenty for most applications, but I guess Erlang could fill use cases where you need 100x that.

            1. 8

              There is nothing inherently “heavy” about WebAssembly. In practice allocating 64Kb/process doesn’t mean much, as most operating systems utilise virtual memory and if you don’t use the memory you don’t pay for it. I’m not sure what you mean with IPC speed, the overhead of sending messages between processes in Lunatic should be as optimised as it gets. Lunatic also uses one of the fastest context switching implementations available. I think that it’s a bit unfair to dismiss a new project just because it can’t compete with one that had decades of optimisations put into it right from the get-go.

              1. 5

                I didn’t dismiss your project, I asked to see a justification of your claims, which I think is a reasonable request. And you provided a justification, which was helpful, thanks.

            2. 3

              My superficial reaction to this is that it sounds more like a goroutine model than an actor model. Which is not bad, but basically more low level (for better and worse).

              Given async/await’s backlash you’re probably arriving at the right time with this.

              1. 2

                Actors aren’t hard to implement on top of (goroutines/ dispatch queues/ etc.) I’ve done it, in C++, as the concurrency substrate for my work project. Mostly you just need lambdas and a reasonably efficient producer/consumer queue thereof.

                The lambdas here might be more like IPC messages since the actors would often be in separate processes. But again, that’s not rocket science, and I know of other Actor libraries that already support it because they support distributed actors.

                (In my experience, Actors and async/await are not an either/or. One of the frustrations of working with my library is that actor messages are one-way, so you have to use callbacks (delivered asynchronously) to get responses, which leads to the familiar callback-hell, or else formalize the response as another actor message sent the opposite direction, which requires the initiator to keep intermediate state data around. I’ve really, really been wishing for C++20 coroutines with their async/await features!

          2. 2

            Love the project. I always wonder why more programming environments don’t implement the equivalent of OTP supervision trees out of the box.

            I recall being baffled a couple years ago when I started writing Go at work and learned that there was no similar concept to the supervisors I was accustomed to coming from Elixir/Erlang.

            I’ve come to the conclusion that the general trend right now is to have the fault tolerance layer abstracted to the infrastructure level with k8s and similar instead of application level. Although in use cases such as wasm that doesn’t make sense of course.