1. 4

    Sorry to go off on a tangent, but it’s unfortunate that they chose to implement their own Lisp-like language rather than write in Lisp in the first place. Writing it from scratch misses the point of Lisp as a DSL.

    Judging by their example code, they could’ve written a few Common Lisp macros and had nearly identical state machine code. Then they could’ve focused entirely on the domain specific parts of the language and piggybacked on CL for the “boring” parts like arithmetic, looping, etc.

    Granted Lisp isn’t very popular, but it’s more so than Radicle and there are many books and tutorials and pre-existing libraries (of varying quality, of course). But now their team will have to support a whole language ecosystem just to support the distributed state machine work they’re actually interested in.

    1. 3

      We needed, for this entire architecture to work, a language that is deterministic. If there’s some compiler extension that restricts CL to only the deterministic parts, that could have worked. But (and I have to admit I’m relatively ignorant about CL here) I’d be surprised if that were easy, and didn’t imply most tutorials and libraries wouldn’t work anyhow.

      1. 2

        I’m not sure I understand the use of “deterministic” here. Which parts of CL are non-deterministic?

        In any case, I used CL as the example because it’s what I use, but a simpler Lisp with macros, like Scheme, should also work.

        1. 4

          By “deterministic” I mean that the language should be such that, for any program written in it P, and for any inputs to that program I, the result of running P with I, P(I), should always be the same between runs, for everyone. So no IO, no platform-specific behavior, no concurrency, no randomness, no weak refs that allow observing garbage collection results, etc.

          1. 2

            The state machines defined using Radicle need to be deterministic, but that doesn’t require the language defining the state machines to be deterministic.

            1. 9

              That’s true, but then that would require a separate proof for each machine (or the risk of a mistake). The idea behind Radicle is a special purpose programming language in which all programs are automatically deterministic state machines.

            2. 1

              Why not just use a pure language?

        2. 2

          chose to implement their own Lisp-like language rather than write in Lisp in the first place.

          I’m not sure it’s from scratch…. “Lisp dialect in the flavor of Black.” combined with the fact Black’s flavour is…

          Black is an extension of Scheme with a reflective construct exec-at-metalevel. It executes its argument at the metalevel where the interpreter that executes user programs is running. There, one can observe, access, and even modify the metalevel interpreter in any way. Because the metalevel interpreter determines the operational semantics of the language, Black effectively allows us to observe, access, and modify the language semantics from within the same language framework.

          ie. I’m curious to know to what extent it’s “implement their own Lisp-like language” vs a “scheme minus mutability” with a Black program that mutates Black.

          After all Scheme isn’t exactly a massive language.

          1. 1

            After all Scheme isn’t exactly a massive language.

            It’s … a fair bit bigger than that. That doesn’t even implement all of r4rs. The most recent standard, r6rs (from 2007), expanded the language so significantly than many (real, non-toy) schemes still haven’t implemented everything.

            I agree with your overall point, just that your example is misleading.

            1. 1

              The most recent standard is r7rs and as I understand it, it makes the language smaller than r6rs.

              1. 1

                I forgot about r7rs. IIRC, that added so many things to the language that they split it into two specs: the big one and the small one (the latter which is closer to r5rs (again, iirc)).

            2. 1

              What I was getting at is the language isn’t written in Lisp or Scheme, but is a Lisp-like language written in Haskell. The power of Lisp as a DSL is that macros allow the language to be seamlessly extended using the language itself. Instead of writing an entire compiler, they could have written a few macros.

              The nondeterminism argument that jkarni and jameshh brought up kind of makes sense, but seems like it would be easy to circumvent by generating Radicle code at runtime.

              1. 1

                Well, the meta-level stuff is interesting and does more than a macro could do…. and not available in the Lispy things I have used…

                I wonder if and what they use it for…

          1. 5

            decentralized state machines? Sign me up!

            1. 2

              I’m not sure that’s what this is… it seems more like the machine is stored in IPFS and is recreated from the entire history of events / state changes / inputs it has received, then serialized and pushed back to IPFS? Or perhaps the machine definition and the sequence of events / inputs are two separate things.

              And there’s also this

              The owner of the pointer is also, in a way, the owner of the Radicle machine. In theory, they can choose which new inputs are accepted (though accepting inputs is an automatic process guided by the semantics of the machine, so under normal conditions this won’t happen). The owner’s radicle daemon subscribes to an IPFS pubsub channel, and anyone who wants to submit new inputs to a machine sends their input to the relevant pubsub channel, so that the owner can add it to IPFS and update the machine’s pointer.

              Note: This means that if the owner is offline, writing won’t work, and the machine becomes read-only.

              Which makes it sound like you can’t use the system if the owner is offline, unless it’s been made public data? It’s not clear either what happens if multiple people try to send different inputs to the state machine at the same time when it’s offline, either - what decides which input is processed first, and what happens if that means that the machine is now in a state where the second input is invalid?

              Probably this has all been thought of and I’m just not understanding it well…

              1. 3

                it seems more like the machine is stored in IPFS and is recreated from the entire history of events / state changes / inputs it has received, then serialized and pushed back to IPFS? Or perhaps the machine definition and the sequence of events / inputs are two separate things.

                Just the input log is stored on IPFS. Each person interacting with it fetches that log and recreates the machine from that. So very much like traditional RSMs. Except that, as mention in the blog post, the code defining the RSM is in the log itself, since we start off from an “ur-RSM”. (And that, of course, the input log is on IPFS so much more decentralized than most RSMs in practice are.)

                Which makes it sound like you can’t use the system if the owner is offline, unless it’s been made public data?

                Sort of. You can still read data from someone’s machine; you just can’t (yet) write to it. So if the machine represents, for instance, the issues for one of your projects, I can see what issues are open even if you’re offline (assuming someone replicated it, which in practice should be nearly always since having replication services like SSB’s Pubs is easy). But I can’t open a new issue, or comment on an existing one. So to answer your question about what happens when multiple people send inputs when the owner is offline - well, nothing. Nothing gets written if the owner is offline.

                That’s of course a serious limitation, of course. We have thought quite a bit about the possible solutions to it, and your points are very important considerations when thinking about those potential solutions. There are quite a few, all with different trade offs. But in general the answer to “which input is processed first” is that indeed, there are fewer guarantees about ordering of messages not yet delivered to the owner, and that indeed this can make an input invalid.

            1. 5

              The Radicle IPFS daemon is the process that runs an IPFS daemon, bootstrapping to our own separate Radicle IPFS network.

              I know next to nothing about IPFS so this is a genuine question: why is the Radicle network separate from the main IPFS network? Does this mean that if I want to use both IPFS and Radicle then I need two daemons?

              1. 4

                For you second question: yes - they run independently of each other.

                They are separated as we don’t replicate general files, you can’t really make use of them without being in the context of radicle. A better way is to think of ipfs as the technology we picked for replication and not the ipfs network which one partakes when starting the default ipfs daemon.

                1. 3

                  The main reason is that currently IPNS name resolutions take a much longer time in a larger network. It seems like we’ll soon have fixes for that, so in theory we could move back to using the main network.

                  1. 2

                    But can’t you use IPFS proper, while just ignoring IPNS for now? I thought the latter is just an extra layer over the former?

                    1. 3

                      We use both - IPFS for storing the data, and IPNS to point to the head of the list/log of expressions to a machine. Thus, when you update a machine (e.g., add a new issue), you’re adding an item to IPFS that has the new data plus a pointer to the old data, and updating the IPNS to point to that. When someone reads, they resolve the IPNS, and then read the corresponding data.

                      In theory we could implement such a pointer system ourselves, or use something other than IPNS. But it seems like improvements to IPNS are happening faster than we could come up with an alternative, so we settled on IPNS.

                      1. 1

                        Ah, so you’re using a separate IPFS network because the normal one is too big for IPNS now, and yours is still at an early stage, thus small enough that IPNS is fast enough? If yes, then please do move to normal IPFS as soon as you find that the improvements make it feasible. And, for the sake of future generations, please don’t use an “It’s too late” argument to avoid such a move. It won’t be.