1. 7

  2. 3

    Supercompilation has several definitions, but none of them are what you’re talking about; in general, supercompilation of a program involves transforming the program into an abstract collection of all of its possible behaviors, and then combining those behaviors into a new “supercompiled” program. This can take exponential time but can be proven to find the smallest/fastest/etc. program which is equivalent to a given input program.

    If a normal compiler compiles to an executable, a supercompiler would compile to a cloud deployment. All of those considerations like database engines, scalability, load-balancing, health checks, certificates, pods, edge-computing strategies, CDNs, and so forth would exist as supercompiler options and config files.

    This sounds like Kubernetes. In Kubernetes, objects are used to represent microservices and also to express the supporting infrastructure which you mention. Microservices are objects in the message-passing paradigm, and Kubernetes treats them as just another type of object which has state to maintain and resources to consume.

    The app must have tests. The app must do what the tests say it does. The app cannot do anything that the tests do not cover. These tests exist in a compilation unit separate from the app itself and are in no way coupled to it.

    Tests examine properties of programs. Thus, Rice’s Theorem is summoned from the depths. There are a few ways to avoid this:

    • Write the app in a non-Turing-complete language like awk, sed, or jq
    • Write infinitely many test cases
    • Write property tests with infinite support and use probability theory to estimate coverage when the tests are run randomly
    • Make the app identical to its own tests, as a sort of trivial proof-carrying program

    But I am curious as to whether you’d have some example code that you are willing to share which already has tests that are written in the way that you discuss here.

    1. 1

      The second book has some, although admittedly not as much as I’d like. It’s exactly what you’re requesting.

      I need to provide a more in-depth example. Over the next month or two I’ll write something up. Part of the difficulty here is scoping the essays. I can write an essay that covers function assembly through DDD/strong typing, I could write an essay that covers the innards of the compiler (which I don’t think is like k8s at all), or I could write an essay about the process of incrementally developing an app. But I can’t write all three at once, and whatever I write has to have enough code so that a reasonable developer can see where it’s going. Otherwise I’d end up having to write a tools suite and enterprise app just to elaborate. That’s a non-starter for me.

      We should continue this when you have a better idea of the concept. Sounds like I have more work to do.

      1. 3

        Otherwise I’d end up having to write a tools suite and enterprise app just to elaborate. That’s a non-starter for me.

        I can’t empathize; I’m literally writing an entire programming language and toolchain because I think that it ought to exist, and it didn’t already exist. To replay the quote at the top of the E programming language homepage:

        We do not influence the course of events by persuading people that we are right when we make what they regard as radical proposals. Rather, we exert influence by keeping options available when something has to be done at a time of crisis.

        Your articles are radical proposals. Thus, they need to come with options which people can choose today; people need to be able to explore your radical way of expressing and maintaining code. This will allow them to choose your techniques when they have greenfield projects in need of design, or other moments of crisis.

        1. 1

          You are correct. Although they’re just extensions of FP, DDD, and a lot of other tech we’ve had for a long time, they’re extensions mixed together in a way that probably isn’t obvious to the average programmer scanning the essay over lunch.

          I am a bit betwixt and between here. Obviously I need to provide more technical detail for more folks to pick it up (I think you’re pretty close). But the tech/language/platform really doesn’t matter. If I pick up a handy language, say Ocaml or F#, and start coding? Then I’m going to lose people who aren’t familiar with the language. The Great Pumpkin only knows what a mess this would be to explain using something like traditional OO. It wouldn’t be a series of essays or a book, it’d be a seminar.

          I think I’ll write an essay with a couple of microservices with stubbed out functions inside. I can then demonstrate how decoupling the types and tests from the execution of the code itself allows us to recompose the same functionality for other purposes, say app-wide testing or making sure outputs and inputs all match. That’d be a start. Maybe it would be a happy medium.

          1. 1

            For anybody following along in this thread and interested in this topic, I spent the morning coding yesterday, drawing diagrams, and framing up this idea in F#. Many thanks to Corbin for reading it and providing feedback.

            I submitted it here. Some bozo marked it as spam and downvoted it. If anybody can explain to me what part of that essay or link was spam I’d love to know so I won’t do that again.

            Just wanted those interested to know that the ideas presented here are a lot more than vague hand-wavy stuff. There’s a there there.

        2. 2

          k8s is one of the compile target arch-OS not the compiler. If I understood your essay the compiler would target something like k8s or just vanilla vms but would not be the compiler itself.

          1. 1

            Yep. There’d be a config file CLI option that would address various cloud options, much the same as there are options in GCC to handle byte alignment.

            1. 2

              Only tangentially related by I have a toy/side project who’s entire purpose is to compile into configuration file formats. A DSL for generating the various configuration file formats that get used to deploy these things. https://github.com/zaphar/ucg

              I imagine something similar would need to be a component of your super compiler.

      2. 2

        An app that cannot crash. It has a way to fail gracefully but that doesn’t involve some downstream coder poking through your call stack.

        Could you elaborate on that? I am not sure, if misunderstand this completely, but looking at badly designed software I remember misleading, confusing, sometimes outright wrong warnings, errors, etc. being logged rather than an actual (honest) crash that would have head me to the root cause to be fixed.

        Also at least some of the mechanisms I have seen were very much tied to “large frameworks” or at least far away from “micro”, like you mention earlier.

        1. 1

          The microservice cannot crash in the same way an SQL statement cannot crash.

          You can hit it with anything, but it takes whatever you give it and transforms it into something it can use or an outgoing stream for some other app.

          The code I provided shows this. Let me know if it needs more explaining.

          ADD: Apologies. This was in the new essay. Brain freeze. https://danielbmarkham.com/outlines-of-a-supercompiler-in-f/