Threads for vindarel

  1. 6

    However, appeals to how e.g. lisps provide some kind of unique advantage in interactive development are wildly overstated (and I say this as a former advocate of such practices).

    Anyone who has used a Forth can confirm that the “person interacting with running program” model is by no means unique to lisps. There are also several other non-lisp languages that have no technical barrier to accomplishing this mode of operation, but typically do not do so solely for cultural reasons; Ruby and Lua in particular come to mind. Then you have systems like Erlang that go far beyond what any lisp can do in terms of rolling out changes to production systems live without a restart, and literally keeping both the old and new version running simultaneously in the same program until the old version is safe to abandon.

    It’s also weird that the “On repl-driven programming” article describes a property of “lisps” that as far as I know, no lisp in the world other than CL supports:

    Define a datatype. I mean a class, a struct, a record type–whatever user-defined type your favorite language supports. Make some instances of it. Write some functions (or methods, or procedures, or whatever) to operate on them. Now change the definition of the type. What happens? Does your language runtime notice that the definition of the type has changed?

    Clojure is notoriously bad at this with the way defrecord instances can persist attached to definitions of the record class that no longer exist, and outdated instantiations will be visually indistinguishable from previous ones but behave differently!

    1. 2

      Anyone who has used a Forth can confirm that the “person interacting with running program” model is by no means unique to lisps. There are also several other non-lisp languages that have no technical barrier to accomplishing this mode of operation, but typically do not do so solely for cultural reasons; Ruby and Lua in particular come to mind.

      I haven’t tracked down the details, but I’ve heard that Wind River managed to make “interacting with a running program” work in C on their VxWorks operating system.

      1. 1

        The shell has a bundled C interpreter.

      2. 1

        As a js developer with hot reload (maintaining state), & chrome dev tools to inspect practically everything & even usually be able to pry into component level state either with a extension or knowing a little bit about framework internals.. what does clojurescript / friends have that im missing?

        1. 2

          The same workflow for ANY program!

          (edit: thinking CL, not necessarily ClojureScript)

          1. 2

            Fair!

      1. 14

        I have one big problem with this talk, and that’s… its central thesis actually: one thing the edit-compile-run cycle has going for it is that it’s principled. Namely, you’re starting from a known state (the source code) every time you compile. With live/image based programming where you fix everything on the go, the actual source code isn’t limited to the source text, but the state of the program itself! This has to be unmanageable at some point.

        I will also note that some aspects of live programming are available even to C/C++ programmers: just recompile the dll and reload it. This requires significant initial setup, but it allowed more than one game dev to change mechanics on the fly without restarting the whole game, without the help of a scripting language. The Yi editor written in Haskell demonstrated something similar. XMonad can also maintain its state across compilations (which happen whenever we change its configuration).

        1. 4

          Well said! I do love REPLs and think they are very valuable, I also prefer “principled” approaches (as you very well put it) and think there is a great deal of value there.

          Combining both seems like fertile ground.

          1. 1

            Exactly, it is about combining both. Principled use of excessive power.

          2. 3

            Yup. There have been some threads here about “image-based” development, which I experienced with Smalltalk-80. It’s kind of magical in some ways, but also leads to real problems because your program never starts from scratch.

            1. 2

              With live/image based programming where you fix everything on the go, the actual source code isn’t limited to the source text, but the state of the program itself! This has to be unmanageable at some point.

              I am a super huge fan of Common lisp and this is something I see every single newcomer struggle with. They usually don’t struggle with it for long though. But it’s definitely something that I would like to see discussed explicitly more often.

              On the bright side, Common lisp was designed with that (interactive development) in mind (mostly). What comes immediately to my mind, as an example of this, is the defparameter v.s. defvar expressions that only differ by what they do when they are re-evaluated. Making it easy to reload source code without losing the current state.

              Yet another thing: I’ve rarely seen anything that deals with “cleaning up the state”. For example, if you rename a function f1 to f2, the original function f1 still exists in the image unless you explicitly remove with (with the so well named function fmakunbound). The only time I’ve seen something that deals explicitly with renames is terraform’s refactor, but that’s hardly for “image based development”.

              One last thing that I won’t develop because this is already way longer than I wanted: the problem of updating the code within an image is the same problem as updating a database.

              1. 2

                the problem of updating the code within an image is the same problem as updating a database.

                I think you meant to analogise the database with the state, not the code…

                There’s a difference though: with a database, the data is concentrated in one place, and that data is mostly statically typed: you have a fairly rigid set of tables, each column has a definite type… and if it’s all normalised like it should be 99.9…% of the time the interface between it and the code will be relatively thin, and very stable.

                Now replace the database with the live state of a long lived program. A dynamically typed program at that. You need to make sure your program state is just as principled as the database’s, the interface between it and the code just as thin and stable… except here all you have is programmer discipline. There’s no clear fence, and depending how your program is accessing your data, good luck determining whether a state change will break your program or not. At least with a database you know that as long as you don’t remove columns or entire tables you’re mostly safe.

                So at a first glance, image based programming at scale (let’s say more than 10 devs working on the same code base & associated state) is looks utterly impossible. How does that work in practice?

                • How do you keep your program from irrecoverably mangling its state?
                • How do you deal with irrecoverably mangled state?
                • How do you version the code?
                • How do you version the state?
                • How do you share state among several collaborators?
                • How do you update the code of a production instance while keeping existing state?
                • How do you update the state of a production instance?
                • How do you update the state architecture of a production instance?

                And perhaps most importantly, do you even need all of the above? Why not?

                1. 3

                  In CL: if the state becomes a mess, restart your image! No big deal, you loose 2 minutes. For many days, I can develop my program with the same running image, it’s fast, it’s interactive, I don’t loose state, I am happy. It happens indeed that I restart it. I also always run unit tests or build the project from the terminal from time to time, so from a blank test.

                  We use source files with git. Maybe they didn’t have this in Smalltalk land decades ago, we do now.

                  How do you share state among several collaborators?

                  To share state among developers, we can build and share a custom image. It’s nearly like an executable, but it’s an image that offers a REPL. Now instead of running a new SBCL instance from our editor, we start sbcl --core my-image, and it starts instantly.

                  How do you update the code of a production instance while keeping existing state?

                  Easy: git pull, connect to the running image (you have a tmux or you use Swank, the server side of Slime, through an SSH tunnel) and you load your source files again (today, it would quickload). The image is not restarted, so you keep state.

                  Also if you define variables with defvar, you don’t loose their state (versus defparameter).

                  But, disclaimer: one can very well update production instances without worrying keeping any state: rsync a new binary, stop the service, start the new service. Usual deployment knowledge would apply. Just as with a Go application for instance.

                  How do you update the state of a production instance?

                  wow, you mean, from the state of a dev instance to the production one? Who would care to do that? I don’t see the point. I develop on my machine, I deploy new code to the production machine. I don’t have to mix states. Maybe for very specific type of applications? If I need to send data, I send the data files.

                  1. 5

                    In CL: if the state becomes a mess, restart your image! No big deal, you loose 2 minutes.

                    Oh, I see. So the people raging about image based programming actually don’t really care about their image. And I guess then state is not versioned either. Except of course whatever is needed to establish an initial state from scratch. I also bet the need for several people to work on the same image ranges from limited to nil.

                    What I thought was a completely alien realm seems actually pretty mundane all of a sudden. What I thought impossible probably is impossible… I just had the wrong idea. Kind of anticlimactic to be honest, but I think I have a better idea of what this really is now.

                    Thanks.

                    1. 2

                      don’t really care about their image

                      That’s a way of putting it :p For me, a young-ish lisper, image-based development is awesome for: development, and ease of live reload (if I want). Bonus, I use custom images from time to time (start a Lisp image with a ton of libraries pre-installed and with other tweaks). That’s it, and that’s a lot already.

                      1. 1

                        Oh, I see. So the people raging about image based programming actually don’t really care about their image.

                        I think it’s different between CL and Smalltalk, maybe? In CL, your image is pretty important during a given development session, but not so much between developers, or between yourself on different days, I think. The main thing is that your changes are applied to a running image without a restart, and you can build up a running program iteratively like that. If you need to get back to a particular state after resetting your image (restarting your Lisp process), you’d set it up the same way you set up fixtures for a unit test in a stop-compile-run language.

                        1. 1

                          Well if some programmers do care about their image to the point they will want to version it in some way, or work on it collaboratively over a long period of time… how they do it remains a total mystery to me.

                          1. 1

                            I think some Smalltalk implementations have had some kind of tooling for versioning images, but I have no experience of them.

                    2. 3

                      the problem of updating the code within an image is the same problem as updating a database. I think you meant to analogise the database with the state, not the code…

                      Yes and no, but I definitely wasn’t clear xD . I had in mind that updating functions (or methods) in an image is somewhat analog to updating views in a database. But anyway, you seem to understand the problem of updating the state of a running image well enough :)

                      So at a first glance, image based programming at scale (let’s say more than 10 devs working on the same code base & associated state) is looks utterly impossible. How does that work in practice?

                      As @vindarel said, people don’t do that, but!

                      1. It’s a problem that I find interesting and think about from time to time. The solution, I think, would be something like “gating” the changes made to the running images. So, one dev could redefine a function (in a sandbox, which I have no idea how that would work exactly), if it passes the tests, then the change is applied to the “real” image. It would practically be a CI-within-a-process.

                      2. People most definitely don’t care about their dev images, but some do care about the real, always running, production image. In which case they go to great length to make sure the code that is going to be evaluated during a release don’t mess up the state, don’t cause any (foreseeable) bugs and don’t pollute the image. I think this is where my analogy with database migrations came: applying successive version of the codebase to a running image is analog to running successive migrations to a database, which brings me to:

                      3. Checks, in Common lisp, you can compile the code before actually evaluating it. SBCL (the most commonly (ha!) used implementation) will catch a lot of “type errors” (which it often reports as warnings, but you can count those warnings as errors). Also, nothing keeps you from running tests for anything you can think of.

                      How do you update the state of a production instance?

                      Imagine you have an ever running python or js repl, how would you go about updating a global variable that contains all the state (to keep it simple)?

                      The first thing that comes to my mind is: only access the state via some specific functions (often called accessors). Update them to support the new and the old state “shape” (I think it’s what you meant by “architecture”) before updating the state.

                      How do you keep your program from irrecoverably mangling its state?

                      I covered it a bit, but you kinda just have to be mindful about the changes to the state; type-checking at compile time, assertion at run-time and automated tests all helps. Also keep in mind that you can (and are very often strongly encouraged) to use functional data structures, which helps a lot when you are wary of mangling the state. BUT, bugs can happen, same as database migrations, it can go wrong even with a lot of precaution.

                      How do you deal with irrecoverably mangled state?

                      As vindarel said, just restart the process :P

                      How do you version the code?

                      Same as with all “mainstream” programming languages. You use git.

                      But, @gcupc is right about Smalltalk. Smalltalk’s source code was traditionally part of the image. Smalltalk was much more “image-based” that Common lisp on that aspect. I don’t know much more about it though… Except perhaps that I know that Pharo, a recent Smalltalk implementation, tries hard to make Smalltalk work with source files, so that people can use “traditional” command line tools on those files.

                      How do you version the state?

                      I never heard anybody doing that exactly. But the “state” would be just a file, so you could use git (or better, git-lfs) to version that file. But I don’t think that’s what you had in mind. (And sbcl is super picky about running an image from a different version, so you would be stuck with a specific version of the lisp implementation.)

                      I think that what you had in mind is “I have a running process, I apply some code change to it, how does the code know what’s the state’s schema right now?”. I see 2 solutions: you can do it implicitly, by using predicates that checks the types at runtime, or you can keep a “global” variable that explicitly keep a version number.

                      How do you share state among several collaborators?

                      Depends what you mean. But in theory, if many collaborators have a running image that they don’t want to close for some reason, they can just git pull the most recent version of the codebase and apply that code to their running image. See point 3, about releasing updates without down-time.

                      How do you update the code of a production instance while keeping existing state? / How do you update the state of a production instance? / How do you update the state architecture of a production instance?

                      See point 3.

                      And perhaps most importantly, do you even need all of the above?

                      No, you don’t need it.

                      Why not?

                      Because it’s freaking hard xD, but I think you already guessed it.

                      / disclaimer: this message is way too long, I’m not gonna proof-read it just now ;)

                1. 26

                  I’m very tired of this “angry person derides everything the industry does and points at very small-scale examples as a better way” thing people often do when giving talks. We should be using notebooks, interactive or visual programming? I’d like to see demonstrations for millions-of-lines codebases. All software should contain a repl to enable live-debugging? I recently tried to download firefox’s debug symbols from debian’s repository, it’s over one gigabyte. If you’re going to make me download more than a gigabyte each time I have to update your software, while almost never needing to debug it, I’m not going to use your software.

                  Pointing at things and saying they’re bad without discussing the constraints that went into building the thing and how they could have been solved differently just makes you sound like an asshole. If your goal is to give a quick overview without being able to go into details, then drop the “this other thing is shit” part of your talk.

                  1. 7

                    Thank you for this comment. I got the same feeling when watching the talk. The content is somewhat interesting but the tone is arrogant and insulting to every person who has different practices or opinions.

                    In particular, the examples are heavily biased towards things that are easy to visualize (either pictures, or small snippets of code that do simple things). It’s a bit like Bret Victor’s talks, where everything just happens to be mostly image oriented. I’d like to see how one visualizes compiler internals, SAT solvers code, etc. because it’s the kind of things I’m interested in, not this fluffy “look at little squares drawn on a canvas” thing.

                    And all the interactive stuff doesn’t come for free. How much code do you need to write to produce the visualizations themselves? How do you debug that code?

                    I also wanted to give a quick try to Glamorous Toolkit, mentioned there, because it looks very interesting. It tooks ages to install, and on my laptop it ran at a horrendous (single digit) FPS count. It was almost laughable. I find it revealing that nowhere in the talk does this person mention performance or memory consumption at all (e.g. in the cell example). The dig at C++ is also uncalled for, imho: it’s a language where people get things done, often semi interactively (in game development, for example; see Dear Imgui for how game devs add interactivity to their code), in an environment where performance and memory consumption matter.

                    One last rant: the tweet “stop the world typecheckers”. No. LSP and type checking inside your editor is interactive too, and I don’t believe that a runtime environment is strictly better or strictly more interactive, since you only interact with a single concrete instance of the program’s state. Typechecking gives you some interactivity on errors that might happen in at least one instance.

                    1. 6

                      the examples are heavily biased towards things that are easy to visualize

                      And the attitude toward program representation is heavily biased toward people who can see and visualize in the first place. Yes, that’s most people. I’m afraid my comment on the orange site had a self-righteous tone to go with that of the talk. It’s just frustrating sometimes when people so completely ignore the fact that not all of us have the same abilities, and what’s constraining to some is liberating and equalizing for others. I had the same frustration with one of Bret Victor’s talks.

                      1. 4

                        I also wanted to give a quick try to Glamorous Toolkit, mentioned there, because it looks very interesting. It tooks ages to install, and on my laptop it ran at a horrendous (single digit) FPS count. It was almost laughable.

                        Glamorous Toolkit is in beta. We should avoid judging it too harshly just because we disagree with the person who recommended it. That would be creating a cycle of dismissal and judgement.

                        1. 3

                          Feenk has been aggressively advertising gtoolkit on Twitter since at least 2019. They seem to think it’s production ready.

                          1. 1

                            Okay, so long as it’s a reaction to the author’s claims and not the claims of a third party. Judge people by their own actions and not how you feel about “their group”.

                          2. 2

                            You’re right. I mean the FPS was almost laughable. I don’t know how hard is it going to be for GToolkit to overcome this level of performance degradation though. I would guess that a GUI toolkit written entirely in a dynamic language might have trouble being as performant as something more, hum, “dead” as the OP would say.

                            1. 3

                              It’s all good! The talk was pretty dismissive and it’s hard not to take a swing at the pillars of their argument. Just want to make sure that we acknowledge that those pillars are other people who might not have as hard of takes.

                              1. 2

                                I appreciate that, thank you :-). It’s easy to be dismissive on the internet.

                              2. 3

                                I would guess that a GUI toolkit written entirely in a dynamic language might have trouble being as performant as something more, hum, “dead” as the OP would say.

                                How about Smalltalk on the original PARC machines (Alto, Dorado, etc.)? If they could do it back then, presumably we can implement a reasonably fast GUI toolkit in a dynamic language on today’s much more powerful machines.

                            2. 2

                              One last rant: the tweet “stop the world typecheckers”.

                              This topic opens up the purpose of development environments. One could seriously argue that there can and should be a difference in quality during exploring code, experimenting, and checking code into the group. The goal of development environment is to speed up the developer while capturing every developer intentions along the way. This often means type checking can be ignored until after the first run.

                            3. 5

                              These sort of talks are about showing alternative futures, not about presenting a complete solution. The talk should be seen to have a different aim. Not every solution has to scale. I teach high schoolers and I’d kill for a better way for my students to inspect their code. Being able to inspect state, see docs for every language construct/keyword in line, see visualizations for built-in collections. That would be tremendously useful. Maybe we can keep expanding the horizon of programs that can be worked with in this way, as hardware gets more powerful or as algorithms/languages get better.

                              Or even think - even a non-scalable toolset that helps you write tricky string-validation functions or regexes or SQL queries - these would be tremendously useful. We already have great tools for inspecting regexes, ofc.

                              I also liked the rapid-fire list of projects to look into. Plenty of jumping-off points for exploring this space.

                              The tone and presentation style may not be great, totally agree. But that’s a presentation layer thing.

                              1. 5

                                In doesn’t have to be that bad. Common Lisp ticks many points: there are millions-of-lines codebases (or have been) (thinking about Google’s ITA Software and SISCOG, both are at least above 1 million lines). You deliver binaries smaller than your image, and commercial implementations (LispWorks, Allegro(?)) can strip out dead code.

                                1. 2

                                  In doesn’t have to be that bad.

                                  I’m fairly certain that in some cases it does, otherwise you’re claiming that every programmer in every niche is misguided for doing things the way they’re doing it instead of your way.

                                  Common Lisp ticks many points: there are millions-of-lines codebases (or have been) (thinking about Google’s ITA Software and SISCOG

                                  Millions-of-lines codebases written in Common Lisp existing does not imply that the people developing these codebases use notebooks, interactive or visual programming. If they do, the orator should have demonstrated the gains they experience by using interactive programming on millions-of-lines pieces of software. This would have made his point stronger, as it would show that interactive programming provides gains across the entirety of the codebase-size spectrum.

                                  I’ll add that the “millions-of-lines codebases” was just one of the dimensions I had in mind. There are other niches like FPGA-programming, real-time programming, high-throughtput programming and many others that would also need to be studied before claiming that there is One True Way of programming that is better for all programming niches.

                                  You deliver binaries smaller than your image, and commercial implementations (LispWorks, Allegro(?)) can strip out dead code.

                                  This is already something all the big pieces of software do. It’s trivial to do with GCC (use -ffunction-sections, and then pass --gc-sections to ld). The reason the binaries are still huge is because compile-time monomorphisation is used very heavily. You might argue that you don’t need to perform monomoprhisation at compile-time and delay it to run-time instead, but then you’re going to lose start-up performance, which is pretty much my point: everything is a trade-off, there is no universal answer and all you can do is work within constraints.

                                  1. 4

                                    Having worked at ITA: the Common Lisp codebase was written using normal text editors, with code checked into version control. Emacs was quite common, and there was some sort of live prompt thing where you could push code updates during development for interactive testing… SLIME maybe?

                                    Last I saw (been many years at this point) it was deployed on SBCL.

                                    1. 1

                                      Very interesting, thanks for sharing your experience!

                                      and there was some sort of live prompt thing where you could push code updates during development for interactive testing

                                      Was the instance of the program running on developer’s machines, or was it running on servers? Was this instance shared among developers? How often would this instance be restarted (if at all?)? Do you know of any video that would demonstrate this workflow, if not on ITA’s codebase, on something similarly large?

                                      I’ve never done much REPL development (aside from debugging JavaScript in the developer’s console) but have always been fascinated by the idea, I’d love to learn more :)

                                      1. 2

                                        This was just a local instance, for debugging/live-testing mostly, is my impression. Production deployment was all the standard “compile from source code, deploy to N machines”.

                                        It’s possible they used REPL for semi-live-development, but it all ended up in normal source code files in version control.

                                        If you want a REPL-y dev experience, you can try out Jupyter notebooks, but keep in mind that people recommend restarting the notebook occasionally (or often!) so state doesn’t get messed up. Basically no one that I know of does pure REPL development for anything but one-off scripts, and for good reason.

                                    2. 2

                                      I didn’t hear claims and I am not making any about One True Way. 2c.

                                      1. 2

                                        I didn’t hear claims about One True Way. 2c.

                                        I think the video’s orator does claim there is One True Way when he claims that compiling ahead of time is bad and that you should really go with repl-programming instead.

                                      2. 1

                                        Millions-of-lines codebases written in Common Lisp existing does not imply that the people developing these codebases use notebooks, interactive or visual programming.

                                        So, I’m kind of a CL newbie, but my understanding is that CL development is generally done semi-interactively. You write your code in a file, which will be under version control, and so on. But instead of stop-the-world, restart-with-changes, you’re normally working in a live image. As you update a function in your file, you run a command in your editor (usually Emacs) to evaluate the updated function, which replaces the existing function definition, and the same for CLOS class definitions - existing objects are updated to the new class definition. You can examine any aspect of the image’s internal state while it’s running.

                                        The difference in CL images and Smalltalk images, as I understand it, is that in CL, your code in your editor can be out of sync with what’s loaded in the image (either because you haven’t evaluated the changes yet, or you’ve reverted to an older version and not evaluated that), whereas in Smalltalk, it can’t; the image is the only source of truth. Because I’m a n00b, I sometimes have to stop my Lisp process and restart it, loading my code from source, in order to be sure the image is in a state I understand.

                                        1. 2

                                          Sounds 100% right to me

                                          Because I’m a n00b, I sometimes have to stop my Lisp process and restart it, loading my code from source, in order to be sure the image is in a state I understand.

                                          People with lot of experience do that too (but not me ofc) , that’s why there’s a command named restart-inferior-lisp

                                    3. 1

                                      True, and there are plenty of examples that could have been used instead. Fetch the Wayback Machine, Sherman!

                                      Sabre C, renamed CodeCenter, was a popular developer tools for about $4K/seat. It included:

                                      • Live debugging. You would be in the middle of loop and want to change the logic. When you typed, the interpreter started, unloaded the currently running object code into the interpreter, let you make the change, and continue stepping though the code.
                                      • Set up displays (arrows and values) of values, all pointers in user space pointing to them, and walk through pointer chains by pointing and clicking.
                                      • Jump to locations where the last pointer to memory was lost and to where it was allocated.
                                      • Do all the usual ‘jump to declaration’ and ‘auto-reformat’ that is common now.

                                      It was used on MLOC systems all the time.

                                      1. 1

                                        The only reference to Saber-C I could find is here and references a last release date of 1999. I did find a video of something named Saber-Plus but it doesn’t show the live-editing capabilities you’re mentionning. Perhaps live-editing of a running program was only practical when building a compiler whose performance is considered “adequate” was still doable for a single small company? I’d imagine building an interpreter for C++’s most recent revisions, able to deal with the intricacies of the object files produces by GCC, Clang or MSVC, that conserves the performances and behavior of the compiler being emulated might be out of reach for lone actors today.

                                        1. 1

                                          Sorry, I didn’t put a date on it. It was very popular in 1990. The company imploded around 1995, partly from competitive pressures from Sun and HP, partly from Sun and HP changing object formats quickly to cause quality issues. Between COFF and the formal specification of ELF, changes occurred which killed the ability to swap in the interpreter, sometime in 1993 IIRC. Added to that, C++ was evolving rapidly from a C-front preprocesser to its own language, and its spec changed constantly.

                                          Oddly, it should not be out of reach. Charging $4K/seat for development tools is out of reach.

                                    1. 3

                                      More advanced than cl-adams, Consfigurator: https://github.com/spwhitton/consfigurator

                                      1. 2

                                        Common Lisp has been n°3 on the TIOBE index in the 80s, if that’s a relevant measure of “popularity”.

                                        CL performance: it does care. CL can be faster than Rust, even run from sources. https://www.reddit.com/r/lisp/comments/owpedp/my_battle_to_beat_common_lisp_and_java_in_rust_on/

                                        ease of implementation

                                        you might be thinking about Scheme?

                                        Python makes very slow, deliberate changes with excellent backwards compatibility

                                        disagree! lol Especially when you look at the ecosystem. For stability (alongside evolution) -> Common Lisp ;)

                                        1. 2

                                          I just managed to build a standalone binary web app in Common Lisp AND consequently how to embed it in a desktop browser window with Neutralino (light Electron). Excited times, expect news, a demo and documentation! (even cooler: it’s about book management) So, I’ll spend more time on it, but I have to come back to client requests and on what brings an income this week…

                                          1. 1

                                            Oooh, I’d love to know more about this someday. How did Common Lisp treat you? What impl did you use? What was your web stack like? etc.

                                            1. 2

                                              Sure, count on me to publish blog posts and write more documentation (especially on the Cookbook). Developing in CL has always been a pleasure. The difficulty in building this was to find how to go around the little warts. There have been… a few. They slowed me down, but they were actually not difficult to solve. Often, I asked on Discord and someone had an answer. I find I didn’t have issues with CL libraries or the implementation themselves, only with all the little inconsistencies and the difficulty of doing something for the first time, and where googling doesn’t throw much results (it does more, now). Example: running my binary with Systemd: SO question. I had to know the answer because the year before I had resolved how to run the webserver from a binary (put it in the foreground: easy, but I had needed to ask). Example: I use Deploy to build the binary and ship a couple foreign libraries. Yes, but it complains about libssl, what to do? It was easy, but undocumented. Another path un-explored was how to ship the Djula HTML templates into the binary. This required cooperation with the maintainer (he’s great). Etc, etc, etc.

                                              My stack: SBCL, Hunchentoot, easy-routes, Djula templates, Mito and sqlite3 for the DB. A bit of HTMX, a bit of JavaScript, very classical web app.

                                              Running the whole with Neutralino was a no-brainer since I have my sandalone binary available.

                                          1. 1

                                            Or we can try a syntax with less parens: Arc’s syntax for CL for example https://github.com/g000001/arc-compat

                                            1. 1

                                              I’m not familiar with Arc or CL, but the examples on that page still have a buttload of parens.

                                            1. 1

                                              I’m not a big Emacs user. I’m not saying that I don’t like it. Just that I don’t use it that much. I keep it there on my task bar as one of the various editors I use for various jobs. One thing though that would make it increase my personal usage of it though, is speed. But I know that interpreting Elisp is not necessarily quick and writing a new interpreter while making sure to maintain compatibility with the tens of thousands of Elisp extension already in the wild would be highly improbable.

                                              1. 3

                                                The latest release of Emacs has JIT compilation of elisp: https://www.emacswiki.org/emacs/GccEmacs

                                                1. 2

                                                  There’s some hope on this front, but I’m not holding my breath - https://www.emacswiki.org/emacs/GuileEmacs

                                                  1. 2

                                                    The code repos in that link haven’t been touched in seven years.

                                                    1. 1

                                                      Hey, you might as well look at Lem: https://github.com/lem-project/lem/ it’s built in CL and it is usable right now, for CL and other languages out of the box. It works on the terminal and it has experimental Electron and OpenGL UIs.

                                                  1. 1

                                                    Nice. He makes me want to try the Shen book instead. Anyone has a feedback?

                                                    1. 1

                                                      And Lisp is still in space with the James-Webb telescope :) (just like Hubbles, the planning and scheduling system)

                                                      1. 1

                                                        … and they stop their hosting service. But open source doesn’t mean end of business. It looks like the author is burned out, but I wonder if they thought about keeping their current users (and not allowing for more if it’s too much work, but redirecting towards (future) alternative hosts). He might enjoy running a side activity with less (financial) pressure.

                                                        1. 5

                                                          Keeping the service running and getting compensated for it means being accountable for support and uptime. I can understand the author wanting to turn it off if he isn’t living off it. That’s actually an opportunity for someone else to offer hosting service I suppose.

                                                        1. 3

                                                          and they launched a PeerTube account :) https://tube.network.europa.eu/

                                                          1. 1

                                                            One thing I really like about Smalltalk is the ability to save and restore the state of the VM at any time at any place. I really miss this in notebook environments like Jupyter as it is impossible to move my computation from one browser window to another.

                                                            1. 2

                                                              Silly idea: use a Jupyter notebook with a Common Lisp kernel and save&restore the Lisp image?? (https://github.com/yitzchak/common-lisp-jupyter)

                                                            1. 5

                                                              Nice. Similar, in Common Lisp:

                                                              • Consfigurator - Lisp declarative configuration management system. You can use it to configure hosts as root, deploy services as unprivileged users, build and deploy containers, and produce disc images. [GPL3][2].
                                                              • Adams - UNIX system administration in Common Lisp. Not unlike Ansible, Chef or Puppet. [ISC][22].
                                                              1. 4

                                                                This list of CL companies is more up to date than the one on common-lisp.net: https://github.com/azzamsa/awesome-lisp-companies/ (it’s updated once in a while, after the latter)

                                                                1. 6

                                                                  This is a blog post from 2018 and it is lacking some stuff. See the more complete https://lispcookbook.github.io/cl-cookbook/scripting.html : it mentions asdf:make (portable equivalent of sbcl:save-lisp-and-die), Deploy, Roswell, then it shows how to parse CLI args, how to setup a CI, and shows a gotcha when building for web apps.

                                                                  1. 2

                                                                    For me this excellently highlights how great lisps are at meeting the needs of an individual, no matter what they’re creating. On the other hand it seems to ignore the collaborative side of development where you’re working on a team of 10 or 20, or maybe even 100+. As the team grows, I think it’s beneficial to have some built in rules (static types, beam error handling, etc) to help guide the team to a common goal. I think it’s also beneficial to use multiple languages in a project, where appropriate. Sometimes thinking in a different language gives you a whole new perspective on the problem(s) you’re solving.

                                                                    1. 3

                                                                      Python has been extremely popular without any static type checking. People went along with unit tests and production errors. So it feels strange to read that Lisps are ignoring something. Moreover, SBCL throws many static type errors and warnings at compile time (when we compile one function with a keystroke). It is already super useful and we could super charge our program with even more compile time checks.

                                                                      1. 3

                                                                        I meant the post ignores collaborative development, not lisp ignores. The post focuses on the individual developer yet ignores that there’s also a lot of collaborative development and sometimes a tool that is excels one area is deficient in another.

                                                                    1. 2

                                                                      You could say most of the same things about Python.

                                                                      1. 10

                                                                        Python’s error recovery and auto-reload is nowhere near as sophisticated as Common Lisp’s. Yes, there’s postmortem debugging (with ipdb and %debug) and %autoreload in IPython, but these are both hacked-on extensions that aren’t as well integrated into the language. A Common Lisp REPL will automatically pause the execution of the program at the point where it died. This allows you to actually fix the state of the object, and continue the computation. Python remembers the callstack when an exception is thrown, sure, but it doesn’t allow you to pick up where it left off directly before throwing the exception. You can place a breakpoint and fix the state before the exception is thrown, but that’s not the same thing.

                                                                        1. 3

                                                                          Run your program with python -m pdb and you’ll drop into a debugger where you can inspect stuff, change it, and restart the computation. All part of stock cpython.

                                                                          1. 1

                                                                            It’s slow, you loose state (=> not truly interactive as in image-based development, the program only stops and restarts) and you are not inside your editor. It is not a first-class experience, but the presence of this switch shows it’s a good thing to have. Moreover, CL has some interactive tools (interactive debugger, stepper, inspector, tracer…).

                                                                            1. 1

                                                                              Image based development is orthogonal to interactive/restart debugging. Image based development is also not a part of the cl standard. For example abcl doesn’t support it.

                                                                          2. 1

                                                                            While not practical, you could implement this in theory with approach similar to pyrasite. (It can do shell injection into a running script) It would be another hacked-on solution, but mentioning it just in case someone wants to have fun.

                                                                          3. 5

                                                                            Sorta, but the interactiveness of Python is more hacky which leaks from the seams sometimes. In Common Lisp I believe it’s a significant part of the core design.

                                                                            1. 1

                                                                              What would you say is the leakiness of pdb? Or the missing strength of the cl restart system?

                                                                              I’ve used pdb a lot and cl only a little. In particular I find nested restarts confusing.

                                                                              1. 3

                                                                                I have to do something to have pdb enabled, and then I’m in a rather different environment from a standard python interpreter. Whereas in common lisp, the debugger/environment is always there to fall on and it’s not a separate part of the system.

                                                                                Can pdb freeze execution during a raised exception, let you change the code and then rerun the same piece again?

                                                                                Granted, even though I code in python quite a lot (possibly more than any other language), I’ve never used pdb – so I’m probably somewhat myopic to its possibilities. I should really learn to use it some day, but so far print debugging has been too easy and effective.

                                                                                1. 2

                                                                                  Can pdb freeze execution during a raised exception, let you change the code and then rerun the same piece again?

                                                                                  Yes, although execution will start from the point where the exception was raised

                                                                          1. 6

                                                                            Quite impressive results from Common Lisp.

                                                                            1. 3

                                                                              and they didn’t even work on optimizing it!

                                                                              1. 10

                                                                                Or, to look at it from a language designer’s perspective, CL and Rust are optimized by design for specific shapes of algorithms and data structures. Because certain chunks of code are idiomatic and common, the compilers are oriented around improving those idioms.