1. 10
  1. 3

    Interesting post, but it also glosses over some important problems: modern development often deals with version control systems, and you can easily go back or forward in time (like when doing git bisect, or when switching branches to do hotfixes and developing new features).

    This completely messes with the “build is a cache of executable code” thing. If you’re using an image-based system, you run a very real risk of not knowing what code is actually running in the image, as opposed to the source code you’ve got checked out. Someone might’ve evaluated a debugging version of some procedure, and then reverted the source code on disk but not in the image. Or one might’ve added a new procedure to the image but forgot to actually add it into source control. Now you have a bigger problem: how to determine if your source code matches the running image?

    I’ve even had this problem in relatively short-running REPL sessions where I’m working on something interactively and sending code to the interpreter from my editor; then I notice some debugging print code being shown which I reverted already in the source code but haven’t re-evaluated the procedure definition.

    In old-school “build” land where most people live, the same problem exists when dependencies aren’t declared correctly between files. You revert a file and “make” triggers a recompile, but there’s another file with a dependency on this file which doesn’t get rebuilt, and now you have inconsistencies between the build and the source code. The obvious solution is to blow away the “cache” and rebuild from scratch. Similarly, in image-based environments you could spin up a clean new image and re-evaluate all the source code to obtain a fresh and up-to-date version that matches the source code on disk. Images just add a layer where you can have parts of files which are outdated. That problem doesn’t exist when “building”.

    1. 2

      However, tests are not themselves part of the build, and need not rely on one - the build is just one way to obtain an updated application. In a live system, the updated application immediately reflects the source code. In such an environment, tests can be run on each update, but the developer need not wait for them.

      This is a really weird statement. In the systems I’ve worked on, the tests are the main point of even having an automated build. If the developer doesn’t wait for the tests to run, then they can’t use the results of the tests to decide what to do next; in that case why even have tests?

      1. 4

        In a “live” environment, which includes service-oriented architectures, it’s impossible to unit-test the world. You have to do integration tests, which ideally includes some testing “in production”. Systems have to be designed in a multi-tenant style, so you don’t have to have an end-to-end test that runs every dependency in a fully isolated stack for every CI run. In this world, you can deploy instantly to a test stack and roll out incrementally to users/clients.

        Consider Stripe’s “test mode”. You can’t run Stripe’s API locally, and it’s a large surface area to mock. Even if you could mock it, what are you really testing anyway? So you run tests against their test mode. Turns out that this works quite nicely. Why wouldn’t it work for your own services? You can have your own test mode, whatever that means for you app/service, and that can go “live” instantly without waiting for any extra checks you need to run prior to deploying that version to customers. A CI job can be kicked off asynchronously to run those integration checks and, if it succeeds, it can update a DNS record or a database entry or something like that.

        Multi-tenancy is the way to rationalize a world in which you can’t control everything. You don’t have an atomic system, so why would you expect to have an atomic build?

        1. 1

          My impression is that the disciplinary approach you have just described and the technical feasibility of running tests in a live environment updated incrementally are orthogonal things. You could wait for the test results and decide what to do next, that should not be a problem.

        2. 1

          It is funny; I read the first paragraph (unbeknownst to the fact that the essay is authored by Gilad Bracha) and all I could think is that builds are not an issue in Smalltalk environments.

          Smalltalk really is a programming environment that changes your perspective about the whole software engineering field. I recommend every programmer to give it a serious try at least once.

          1. 1

            How much is “a serious try”?

            1. 3

              It is hard to tell, but it’s not about learning the syntax, writing FizzBuzzes and the like. Having a basic understanding of a Smalltalk system on the architectural level is the important thing in my opinion. After that, when you are solving a nasty architectural problem in another environment, think for a while on how Smalltalk would help you solve it. It is not a far cry from what is considered to be the UNIX way: if you avoid writing monoliths and you use IPC to coordinate software components, you will notice that Smalltalk is the next iteration of that approach. But the personal computer revolution kind of got in the way and OOP was vandalized.

              From a personal perspecitve, I would really like to see what the next iteration of that would look like, and I have been spending a considerable amount of time thinking about that. Unfortunately I couldn’t come up with an answer yet.

              1. 1

                I don’t edit the code on the production server, so how do you distribute a program in an image based system like Smalltalk? The environment I work in is not the web. The product I work on lives on the phone network, and if it’s not running, not only are we not getting paid, but we’re the one paying our customer (in this case, the Monopolistic Phone Company). The SLAs are scary and if I stop and think about it too much, I want to throw up over how much I could cost the company I work for if my code doesn’t work. And I don’t see how Smalltalk can actually work in such a situation.

                1. 3

                  I am almost sure that I did not understand your point, but I will answer it nonetheless.

                  Generically speaking, there is no impediment in treating an image-based system as an ordinary monolithic, build-based system; the opposite is not true.

                  I know nothing about your problem domain, but in a hypothetical scenario where the state-of-the-art approach to computing are image-build systems, you would still be able to distribute specific versions of images that wouldn’t receive live updates.

              2. 3

                Sometimes I question how much I would gain from learning Smalltalk. I think it has three main lessons of value:

                • What it’s like to code in an environment where everything can be rewritten at runtime, using itself. <- This one is huge! Everyone should learn it, because it’s just so rewarding and refreshing. But you can learn it a lot more practically in Elisp than in Smalltalk.
                • How message-passing works. <- Not nearly as important as the previous one, but still pretty neat. But again, more practical to learn this thru Erlang or Elixir nowadays.
                • What it’s like to work in a “turtles-all-the-way-down” system. <- I don’t really know any modern substitute for this, except that you can get a glimpse at how great it must be by using modern systems that aren’t like that, getting frustrated, and imagining the opposite.

                But maybe there’s more, and I’m missing out? I would say that #1 above is by far the most important, and I think learning Smalltalk (which I can’t use at my day job) would just lead to more frustration vs learning Emacs, which I can and do use every day.

                1. 2

                  Yes, the first point is the most compelling, and it is so blatant to me that our current paradigm is such a haphazard attempt to achieve it that it is not even funny. It gives the impression that monolithic programs that encapsulate all necessary components and are pretty much fixed are the right way to write complex applications.

                  But those communicate with other applications using an API which is usually REST-based, pretty much oblivious to the fact that the other endpoints may receive updates over which you have no control. And you either encapsulate every single application in things such as containers or you run those on dedicated boxes because you know that hell would break loose if you didn’t do so.

                  But if you play the long game you will realize that, in production, those environments are expected all of them to run long uptimes, with high availability. So what is static in a time frame of two hours is extremely dynamic if you look at a server with an uptime of months of even years.

                  So it is all a matter of scale, either time or, say, the applications/MB ratio.

                  1. 1

                    We haven’t quite figured out good composition models that scale up. Most of the infra tools are attempts to patch over this. In the medium-small we have compilers that validates, statically binds and outputs a binary in one shot. Larger than that we have code and config that spins components up/down and binds them dynamically. There’s a whole hodgepodge of patterns to create, verify and exercise these bindings.