1. 2

    I’d really like to write more D. In my particular case, I couldn’t have a GC in play (self-imposed memory constraints), but there’s a lot about it that’s attractive to me. I don’t have any desire to choose Go over it - power of the language is considerably greater from my limited experience.

    That said, Go does have a big package community behind it like Rust.

    1. 14

      Stick a @nogc on your main function and you have a compile-time guarantee that no GC allocations will happen in your program.

      1. 5

        Neat - I didn’t realize this. Too late now for the current project, but good to know for the future. I’m particularly interested in its C++ FFI story. There’s a couple of specialized C++ libraries I’d like to use without having to write flat-C style wrappers just to call them sanely from Rust.

        Thanks for that!

        1. 2

          That’s exactly the kind of tip I was hoping for in the comments. Thanks!

          1. 4

            It always the same arguments with D discussions:

            • I don’t like D that has a GC!
            • Just use @nogc!
            • But then some stuff from the standard library does not work anymore!
            • How much of the standard library?
            • Nobody knows and how would you measure it anyway?
            1.  

              It’s at least a pattern that’s solvable. Someone just has to attempt to compile the whole standard library with no GC option. Then, list the breakage. Then, fix in order of priority for the kind of apps that would want no-GC option. Then, write this up into a web page. Then, everyone shares it in threads where pattern shows up. Finally, the pattern dies after 10-20 years of network effects.

              1.  

                People are doing that. Well, except for the “write this up into a web page” part. I guess you are thinking of web pages like http://www.arewewebyet.org/

                1.  

                  Yeah, some way for people to know that they’re doing it with what level of progress. Good to know they’re doing it. That you’re the first to tell me illustrates how a page like that would be useful in these conversations. People in D camp can just drop a link and be done with it.

        2. 2

          I find D has a lot of packages too. Not an explosive smörgåsbord, but sufficient for my purposes.

          https://code.dlang.org/

          The standard library by itself is fairly rich already.

          https://dlang.org/phobos/

          1. 1

            I guess the question would be whether unsafe or smart pointers are about as easy to use in D as C or C++. If so, the GC might not be a problem. In some languages, GC is really hard to avoid.

            Maybe @JordiGH, who uses D, can tell us.

            1. 5

              I write D daily. Unsafe pointers work the same as in C or C++. I wrote a GC-less C++-like smart pointer library for D. It’s basically std::unique_ptr and std::shared_ptr, but no std::weak_ptr because 1) I haven’t needed it and 2) One can, if needed, rely on the GC to break cycles (although I don’t know how easy that would be do to currently in practice.

              1. 1

                D is a better C++, so pointers easier to use than C++. As I understand, the main problem is that it used to be the case that the standard library used GC freely, making GC hard to avoid if you used the standard library. I understand there is an ongoing effort to clear this but I don’t know the current status.

                1. 3

                  It depends on which part of the standard library. These days, the parts most often used have functions that don’t allocate. In any case it’s easy to avoid by using @nogc.

            1. 4

              Nice article. How do you feel about the size of the language? One thing that keeps me off from looking at rust seriously is the feeling that it’s more of a C++ replacement (kitchen & sink) vs a C replacement.

              The Option example feels a bit dropped off too early, you started by showing an example that fails and then jumped to a different code snippet to show nicer compiler error messages without ever going back and showing how the error path is handled with the Option type.

              You should also add Ada to the list of your languages to explore, you will be surprised how many of the things you found nice or interesting were already done in the past (nice compiler errors, infinite loop semantics, very rich type system, high level language yet with full bare metal control).

              1. 2

                Thank you for commenting! I agree that Rust’s standard library feels as big as C++‘s, but I haven’t been too bothered by the size of either one. To quote Bjarne Stroustrup’s “Foundations of C++” paper, “C++ implementations obey the zero-overhead principle: What you don’t use, you don’t pay for [BS94]. And further: What you do use, you couldn’t hand code any better.” I haven’t personally noticed any drawbacks of having a larger standard library (aside from perhaps binary size constraints, but you would probably end up including a similar amount of code anyway, just code that you wrote yourself), and in addition to the performance of standards-grade implementations of common data structures, my take on it is that having a standardized interface to them improves readability quite a bit - when you go off to look through a codebase, the semantics of something like a hashmap shouldn’t be surprising. It’s a minor draw, but I feel like I have to learn a new hash map interface whenever I go off to grok a new C codebase.

                I’ll definitely take a look at Ada, seems like a very promising language. Do you have any recommendations for books? I think my friend has a copy of John Barnes’ Programming in Ada 2012 I can borrow, but I’m wondering if there’s anything else worth reading.

                Also, thank you for pointing out the issue with the Option example, I’ll make an edit to the post at some point today.

                1. 5

                  It’s funny how perspectives change; to C and JavaScript people, we have a huge standard library, but to Python, Ruby, Java, and Go people, our standard library is minuscule.

                  1. 2

                    I remember when someone in the D community proposed to include a basic web server in the standard library. Paraphrased:

                    “Hell no, are you crazy? A web server is a huge complex thing.”

                    “Why not? Python has one and it is useful.”

                  2. 2

                    What you don’t use, you don’t pay for [BS94]

                    That is true however you have little impact on what others use. Those features will leak into your code via libraries or team mates using features you might not want. Additionally when speaking about kitchen & sink I didn’t only mean the standard library, the language itself is much larger than C.

                    I think my friend has a copy of John Barnes’ Programming in Ada 2012 I can borrow, but I’m wondering if there’s anything else worth reading.

                    Last I did anything related to Ada was somewhere around 2012. I recall the Barnes books were well regarded but I don’t know if that changed in any significant way.

                    For casual reading the Ada Gems from Adacore are fun & informing reads.

                    1. 2

                      I’ll definitely take a look at Ada, seems like a very promising language. Do you have any recommendations for books? I think my friend has a copy of John Barnes’ Programming in Ada 2012 I can borrow, but I’m wondering if there’s anything else worth reading.

                      I recommend Building High Integrity Applications in SPARK. It covers enough Ada to get you into the meat of SPARK (the compile time proving part of Ada) and goes through a lot of safety features that will look familiar after looking at Rust. I wrote an article converting one of the examples to ATS in Capturing Program Invariants in ATS. You’ll probably find yourself thinking “How can I do that in Rust” as you read the book.

                  1. 2

                    The whole thing is great, but one idea that seems particularly useful in arbitrary languages without regard to how it fits with other features is to specify the list of globals used by a function. In Python-like syntax, imagine this:

                    def draw_quad(origin, left, up) [m]:
                        m.Position3fv(origin - left - up)
                        m.Position3fv(origin + left - up)
                        m.Position3fv(origin + left + up)
                        m.Position3fv(origin - left + up)
                    

                    Now you get a guarantee that the function uses zero globals besides m.

                    1. 4

                      In PHP you have something like that, global variables are not accesible from inside functions unless you specifically allow each one you want

                      $m = new M();
                      function draw_quad($orgin, $left, $up){
                          global $m; // or $m = $_GLOBALS['m'];
                          $m->Position3fv($origin - $left -$up);
                          $m->Position3fv($origin + $left - $up);
                          $m->Position3fv($origin + $left + $up);
                          $m->Position3fv($origin - $left + $up);
                      

                      in practice, I haven’t found useful global variables other then the contextual ones ($_POST. $_GET, $_SESSION), which are superglobal and always defined

                      1. 2

                        I’d like to see something similar but generalised to “contextual” environmental bindings, rather than traditional global vars. And a compiler that ensures that somewhere in all call chains the binding exists. But you might want a global in some cases, or a “threadlocal”, or an “import” of some sort, or something like the context in react, etc.

                        Some mechanism in which the compiler makes sure the environmental dependencies are fulfilled, without necessarily requiring that value be propagated explicitly through each owner/container between provider and consumer.

                        1. 4

                          I can’t find it but Scala has an extra context var that is passed with function invocation.

                          And early Lisps had dynamic scope, meaning that a var bound to the next occurrence up the call stack.

                          Both of these mechanisms supply the building blocks for AoP, so that a programmer can mixin orthogonal properties.

                          1. 3

                            And early Lisps had dynamic scope, meaning that a var bound to the next occurrence up the call stack.

                            Today they still have it - see DEFVAR and DEFPARAMETER in Common Lisp.

                            1. 2

                              I can’t find it but Scala has an extra context var that is passed with function invocation.

                              In Scala you can use implicit parameters for this:

                              def foo(origin: Vec3, left: Vec3, up: Vec3)(implicit m: Context) {
                                  m.Position3fv(origin - left - up)
                                  m.Position3fv(origin + left - up)
                                  m.Position3fv(origin + left + up)
                                  m.Position3fv(origin - left + up)
                              }
                              

                              In Haskell you could use a reader/writer/state monad thingy. In Koka or Eff you could use effects.

                              1. 2

                                Yeah, Scala’s context is probably closest to what I’m thinking of, from what I know of it.

                              2. 4

                                You can kinda get this with effect types. Effect types let you label certain functions as using resource A or B, and then you can have a documentation mechanism for what dependencies are used, without passing stuff around.

                                It can still get a bit heavy (at least it is in Purescript), but less so than dependency injection

                              3. 1

                                A compiler or analyzer should be able to tell you that just from what variables or expressions go into the function. A previously-declared global would be one of the arguments. Why do we need to declare it again in the function definition?

                                1. 1

                                  See my final sentence. “Now you get a guarantee that the function uses zero globals besides m.” The program documents/communicates what it uses. The compiler ensures the documentation is always up to date.

                                  In my example, m is not one of the arguments. Because m is a global, lexically accessible anyway in the body of the function. There’s no redundancy here.

                                  1. 0

                                    I’m saying a compiler pass should be able to do the same thing without a language feature. I think it won’t be important to that many people. So, it will probably be optional. If optional, better as a compiler pass or static analysis than a language feature. It might also be an easy analysis.

                                    1. 3

                                      You’re missing the point. It’s a list of variables that the code can access anyway. What would a compiler gain by analyzing what globals the function accesses?

                                      There are many language features that help the compiler do its job. This isn’t one of them. The whole point is documentation.

                                      (Yes, you could support disabling the feature, using say syntax like [*]. C++ has this. Just bear in mind that compilers also don’t require code comments, and yet comments are often a good idea. SImilarly, even languages with type inference encourage documenting types in function headers.)

                                2. 1

                                  What if the Position3fv method uses global variable n? You also need to specify that for draw_quad. This quickly blows up like checked exceptions in Java and people want shortcuts.

                                  1. 2

                                    The problem with checked exceptions is that they discourage people from using exceptions. But there’s no problem in discouraging people from using global variables.

                                    I don’t mean to imply that I want all code to state what it needs all the time. In general I’m a pretty liberal programmer. It just seems like a good idea to give people the option to create “checkable documentation” about what globals a piece of code requires.

                                1. 21

                                  All the stack based ones: Forth, Cat, PostScript, etc

                                  1. 2

                                    I only recently saw my first example of a stack-based language. My thought was that it seems terribly difficult, as a programmer, to keep in mind what the stack contains at any given point in time. Is that something one gets used to over time?

                                    1. 8

                                      I found it fun, when learning Forth, to actually work things out using a physical stack of index cards and a pencil. But yeah, you get used to it pretty quick.

                                      1. 2

                                        In my experience, words rarely cause more than 7 changes to the stack (like, rot pops 3 and pushes them back in a different order, for 6 changes, while dup pops once and pushes the same thing twice for a delta of one change), so if you get used to chunking in terms of only what a word pops and pushes, you can almost treat it like imperative with implicit args.

                                      2. 1

                                        I think it is difficult at first but once you get used to use combinator words (common in e.g. Factor) that probably becomes just as obvious as using map and fold in functional languages.

                                      3. 1

                                        Forth may omit variable names, but makes up for it with many word names.

                                      1. 1

                                        It is P2P which makes it battery draining on smartphones, doesn’t it?

                                        1. 8

                                          “Not only that, any code examples in rustdoc are treated like tests and get executed during compilation!”

                                          This is brilliant. First time I’ve heard of it. ZeD on Hacker News said Python can do it, too, with seangransee offering this example.

                                          Aside from a good idea, it looks like it should also be a new requirement for high-assurance systems where all examples of system usage are executable. It’s often implicit where you’re already supposed to have both tests of all functions and consistency with documentation. At the least, it could be an extra check on these.

                                          1. 14

                                            I’m over the moon about doctests. Elixir has them too, that’s where I saw the light. In the past, I’ve gone to annoying lengths to ensure that code examples in documentation don’t go stale; it’s a great feeling to have first-class support for that workflow.

                                            1. 2

                                              I’ve found that in Elixir having both doctests and normal tests is quite handy–one as a quick sanity check and a way of demonstrating an API, and the other as a full way of confirming behavior. The use of tags to turn on and off different sets of tests is also not well supported (to my knowledge) with doctests.

                                              1. 3

                                                AFAIK turning Elixir doctests on and off is not supported. That bothers me just a bit, because there are times when I’d like to show some non-testable code (e.g., demonstrating a request to an external network service) in the exact same syntax that my doctests use (iex>, etc).

                                            2. 5

                                              I think the first place I saw that was R, where it’s used pervasively in the package repository (CRAN). In fact the output of example REPL sessions is generated by running the code, so besides being used as tests that are flagged if they fail to run entirely, it also keeps the examples up to date with minor changes to output format, etc., which can otherwise get out of date even when the code continues to work.

                                              1. 4
                                                1. 4

                                                  I’ve found that in practice I write far fewer testable Go examples than I do in Rust code. In Rust, I just drop into Markdown, write the code, and I’m done. It’s incredibly low friction.

                                                2. 2

                                                  D achieves the same the other way round. Instead of treating documentation like code, D has unit test code blocks and can treat them as documentation. This makes it easier for the tooling, since syntax highlighting etc applies to the unit tests as well.

                                                  This is not a really critical difference, but if you design a new language, I would suggest the D approach as slightly better.

                                                  1. 2

                                                    I’ve generated documentation from tests before (in Ruby’s rspec). Just thinking about it, isn’t D’s approach prone to the same weakness as the comments that they use to generate the docs? That is, when you use comments or non-executable code (in my case it was string descriptors) to generate docs, they can get out of date with the actual code, whereas if you write tests inside documentation, the docs may grow stale but the tests will break.

                                                1. 3

                                                  Just to make sure: Everybody knows about pushd/popd?

                                                  I tried stuff like CDPATH, but removed it. It can break crappy Makefiles.

                                                  1. 1

                                                    my favorite part about pushd/popd is using dirs -v to see the stack, and cd ~# where # is the directory’s place in the stack…

                                                  1. 35

                                                    I’ll bite.

                                                    General industry trends
                                                    • (5 years) Ready VC will dry up, advertising revenue will bottomout, and companies will have to tighten their belts, disgorging legions of middlingly-skilled developers onto the market–salaries will plummet.
                                                    • (10 years) There will be a loud and messy legal discrimination case ruling in favor of protecting political beliefs and out-of-work activities (probably defending some skinhead). This will accelerate an avalanche of HR drama. People not from the American coasts will continue business as usual.
                                                    • (10 years) There will be at least two major unions for software engineers with proper collective bargaining.
                                                    • (10 years) Increasingly, we’ll see more “coop” teams. The average size will be about half of what it is today, organized around smaller and more cohesive business ideas. These teams will have equal ownership in the profits of their projects.
                                                    Education
                                                    • (5 years) All schools will have some form of programming taught. Most will be garbage.
                                                    • (10 years) Workforce starts getting hit with students who grew up on touchscreens and walled gardens. They are worse at programming than the folks that came before them. They are also more pleasant to work with, when they’re not looking at their phones.
                                                    • (10 years) Some schools will ban social media and communications devices to promote classroom focus.
                                                    • (15 years) There will be a serious retrospective analysis in an academic journal pointing out that web development was almost deliberately constructed to make teaching it as a craft as hard as possible.
                                                    Networking
                                                    • (5 years) Mesh networks still don’t matter. :(
                                                    • (10 years) Mesh networks matter, but are a great way to get in trouble with the government.
                                                    • (10 years) IPv6 still isn’t rolled out properly.
                                                    • (15 years) It is impossible to host your own server on the “public” internet unless you’re a business.
                                                    Devops
                                                    • (5 years) Security, cost, and regulatory concerns are going to move people back towards running their own hardware.
                                                    • (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.
                                                    • (15 years) There will still be work available for legacy Rails applications.
                                                    Hardware
                                                    • (5 years) Alternative battery and PCB techniques allow for more flexible electronics. This initially only shows up in toys, later spreads to fashion. Limited use otherwise.
                                                    • (5 years) VR fails to revitalize the wounded videocard market. Videocard manufacturers are on permanent decline due to pathologies of selling to the cryptobutts folks at expense of building reliable customer base. Gamers have decided graphics are Good Enough, and don’t pay for new gear.
                                                    • (10 years) No significant changes in core count or clock speed will be practical, focus will be shifted instead to power consumption, heat dissipation, and DRM. Chipmakers slash R&D budgets in favor of legal team sizes, since that’s what actually ensures income.

                                                    ~

                                                    I’ve got other fun ones, but that’s a good start I think.

                                                    1. 7

                                                      (5 years) Security, cost, and regulatory concerns are going to move people back towards running their own hardware.

                                                      As of today, public cloud is actually solving several (and way more than people running their own hardware) of these issues.

                                                      (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.

                                                      Containers are actually solving some real problems, several of them already were independently solved, but containers bring a more cohesive solution.

                                                      1. 1

                                                        Containers are actually solving some real problems, several of them already were independently solved, but containers bring a more cohesive solution.

                                                        I am interested, could you elaborate?

                                                        1. 1

                                                          The two main ones that I often mention in favor of containers (trying to stay concise):

                                                          • Isolation: We previously had VMs on a virtualization level but they’re heavy, potentially slow to boot and obscure (try to launch xen and manage vms your pet server), and jail/chroot are way harder to setup and specific to each of your application and do not allow you to restrict resources (to my knowledge).
                                                          • Standard interface: Very useful for orchestration as an example, several tool existed to deploy applications with an orchestrator, but it was mostly executables and suffered from the lack of isolation. Statically compiling solved some of theses issues, but not every application can be.

                                                          Containers are a solution to some problems but not the solution to everything. I just think that wishing they weren’t there, probably means the interlocutor didn’t understand the benefits of it.

                                                          1. 2

                                                            I just think that wishing they weren’t there, probably means the interlocutor didn’t understand the benefits of it.

                                                            I’ve been using FreeBSD jails since 2000, and Solaris zones since Solaris 10, circa 2005. I’ve been writing alternative front-ends for containers in Linux. I think I understand containers and their benefits pretty well.

                                                            That doesn’t mean I don’t think docker, and kubernetes, and all the “modern” stuff are not a steaming pile, both the idea and especially the implementation.

                                                            There is nothing wrong with container technology, containers are great. But there is something fundamentally wrong with the way software is deployed today, using containers.

                                                            1. 1

                                                              But there is something fundamentally wrong with the way software is deployed today, using containers.

                                                              Can you elaborate? Do you have resources to share on that? I feel a comment on Lobsters might a be a bit light to explain such a statement.

                                                            2. 1

                                                              You can actually set resource isolation on various levels; classic Unix quotas, priorities (“nice” in sh) and setrusage() (“ulimit” in sh); Linux cgroups etc. (which is what Docker uses, IIUC); and/or more-specific solutions such as java -Xmx […].

                                                              1. 2

                                                                So you have to use X different tools and syntax to, set the CPU/RAM/IO/… limits, and why using cgroups when you can have cgroups + other features using containers? I mean, your answer is correct but in reality, it’s deeply annoying to work with these at large scale.

                                                                1. 4

                                                                  Eh, I’m a pretty decent old-school sysadmin, and Docker isn’t what I’d consider stable. (Or supported on OpenBSD.) I think this is more of a choose-your-own-pain case.

                                                                  1. 3

                                                                    I really feel this debate is exactly like debates about programming languages. It all depends of your use-cases and experience with each technologies!

                                                                    1. 2

                                                                      I’ll second that. We use Docker for some internal stuff and it’s not very stable in my experience.

                                                                      1. 1

                                                                        If you have <10 applications to run for decades, don’t use Docker. If you have +100 applications to launch and update regularly, or at scale, you often don’t care if 1 or 2 containers die sometimes. You just restart them and it’s almost expected that you won’t reach 100% stability.

                                                                        1. 1

                                                                          I’m not sure I buy that.

                                                                          Out testing infrastructure uses docker containers. I don’t think we’re doing anything unusual, but we still run into problems once or twice a week that require somebody to “sudo killall docker” because it’s completely hung up and unresponsive.

                                                                          1. 1

                                                                            We run at $job thousands of container everyday and it’s very uncommon to have containers crashing because of Docker.

                                                              2. 1

                                                                Easier local development is a big one - developers being able to quickly bring up a full stack of services on their machines. In a world of many services this can be really valuable - you don’t want to be mocking out interfaces if you can avoid it, and better still is calling out to the same code that’s going to be running in production. Another is the fact that the container that’s built by your build system after your tests pass is exactly what runs in production.

                                                            3. 7

                                                              (5 years) VR fails to revitalize the wounded videocard market. Videocard manufacturers are on permanent decline due to pathologies of selling to the cryptobutts folks at expense of building reliable customer base. Gamers have decided graphics are Good Enough, and don’t pay for new gear.

                                                              While I might accept that VR may fail, I don’t think video card companies are reliant on VR succeeding. They have autonomous cars and machine learning to look forward to.

                                                              1. 2

                                                                (10 years) No significant changes in core count or clock speed will be practical, focus will be shifted instead to power consumption, heat dissipation, and DRM. Chipmakers slash R&D budgets in favor of legal team sizes, since that’s what actually ensures income.

                                                                This trend also supports a shift away from scripting languages towards Rust, Go, etc. A focus on hardware extensions (eg deep learning hardware) goes with it.

                                                                1. 1

                                                                  (10 years) Containers will be stuck in Big Enterprise, and everybody else will realize they were a mistake made to compensate for unskilled developers.

                                                                  One can dream!

                                                                  1. 2

                                                                    Would you (or anyone) be able to help me understand this point please? My current job uses containers heavily, and previously I’ve used Solaris Zones and FreeBSD jails. What I see is that developers are able to very closely emulate the deployment environment in development, and don’t have to do “cross platform” tricks just to get a desktop that isn’t running their server OS. I see that particular “skill” as unnecessary unless the software being cross-platform is truly a business goal.

                                                                    1. 1

                                                                      I think Jessie Frazelle perfectly answer to this concern here: https://blog.jessfraz.com/post/containers-zones-jails-vms/

                                                                      P.S.: I have the same question to people that are against containers…

                                                                  2. 1

                                                                    (5 years) Mesh networks still don’t matter. :( (10 years) Mesh networks matter, but are a great way to get in trouble with the government.

                                                                    Serious attempts at mesh networks basically don’t exist since the 200#s when everyone discovered it’s way easier to deploy an overlay net on top of Comcast instead of making mid-distance hops with RONJA/etc.

                                                                    It would be so cool to build a hybrid USPS/UPS/Fedex batch + local realtime link powered national scale network capable of, say, 100mB per user per day, with ~ 3 day max latency. All attempts I’ve found are either very small scale, or just boil down to sending encrypted packets over Comcast.

                                                                    1. 1

                                                                      Everyone’s definition of mesh different, but today there are many serious mesh networks, the main ones being Freifunk and Guifi

                                                                    2. 1

                                                                      (10 years) There will be at least two major unions for software engineers with proper collective bargaining.

                                                                      What leads you to this conclusion? From what I hear, it’s rather the opposite trend, not only in the software industry…

                                                                      (5 years) All schools will have some form of programming taught. Most will be garbage.

                                                                      …especially if this is taken into account, I’d argue.

                                                                      (10 years) Some schools will ban social media and communications devices to promote classroom focus.

                                                                      Aren’t these already banned from schools? Or are you talking about general bans?

                                                                      1. 1

                                                                        I like the container one, I also don’t see the point

                                                                        1. 1

                                                                          It’s really easy to see what state a container is in because you can read a 200 line text file and see that it’s just alpine linux with X Y Z installed and this config changed. On a VM it’s next to impossible to see what has been changed since it was installed.

                                                                          1. 3

                                                                            ate a container is in because you can read a 200 line text file and see that it’s just alpine linux with X Y Z in

                                                                            I just check the puppet manifest

                                                                            1. 2

                                                                              It’s still possible to change other things outside of that config. With a container having almost no persistent memory if you change something outside of the dockerfile it will be blown away soon.

                                                                          2. 1

                                                                            Containers wont be needed because unikernels.

                                                                          3. 1

                                                                            All schools will have some form of programming taught. Most will be garbage.

                                                                            and will therefore be highly desirable hires to full stack shops.

                                                                            1. 1

                                                                              I would add the bottom falling out of the PC market, making PCs more expensive as gamers and enterprise, the entire reason why it still maintains economies of scale, just don’t buy new HW anymore.

                                                                              1. 1

                                                                                I used to always buy PCs, but indeed the last 5 years I haven’t used a desktop PC.

                                                                                1. 1

                                                                                  If it does happen, It’ll probably affect laptops as well, but desktops especially.

                                                                              2. 1

                                                                                (5 years) All schools will have some form of programming taught. Most will be garbage.

                                                                                My prediction: Whether the programming language is garbage or not, provided some reasonable amount of time is spent on these courses we will see a general improvement in the logical thinking and deductive reasoning skills of those students.

                                                                                (at least, I hope so)

                                                                              1. 11

                                                                                Your browser could do the preloading, AMP or not. I believe they tried it once and it breaks a lot of crappy Enterprise intranet apps. For that to work web sites must respect REST: no state changes for Get requests.

                                                                                1. 2

                                                                                  Browsers don’t preload links by default to avoid that sort of breakage, but there are rel=preload and rel=prefetch tags that pages can use to direct that behavior, which is supported by at least Chrome and Firefox. The AMP documentation seems to suggest that AMP gets its preloading behavior (in part? entirely?) by auto-inserting these tags in appropriate places. Is there something else I’m missing here for why other pages/frameworks couldn’t also insert these tags to get similar behavior?

                                                                                1. 2

                                                                                  An IR is primarily a paradigm for compiler engineers. You can port optimizations to different IRs, but it shapes the thinking of the developer.

                                                                                  I have used FIRM a lot and always viewed it as a graph of operations and data dependencies. When I look at a C program, in the back of my mind its FIRM representation appears.

                                                                                  1. 9

                                                                                    I think “lisper” on the HN version of this article had a great summary:

                                                                                    “What’s really going on is that, in Lisp, code is a particular kind of data, specifically, it’s a tree rather than a string. Therefore, some (but not all) of the program’s structure is represented directly in the data structure in which it is represented, and that makes certain kinds of manipulations on code easier, and it makes other kinds of manipulations harder or even impossible. But (and this is the key point) the kinds of manipulations that are easier are the kind you actually want to do in general, and the kind that are harder or impossible are less useful. The reason for this is that the tree organizes the program into pieces that are (mostly) semantically meaningful, whereas representing the program as a string doesn’t. It’s the exact same phenomenon that makes it easier to manipulate HTML correctly using a DOM rather than with regular expressions.”

                                                                                    1. 3

                                                                                      I don’t think tree vs string is the difference. For all languages, it is a string of bytes on disk and a tree once parsed in memory. Lisp just has a closer correspondence between tree and string, which makes it cognitively easier. I don’t know where somebody would draw the line that they are considered “homo”, equal.

                                                                                      1. 6

                                                                                        I think at the point that the tree-literal syntax is the same as the language-proper syntax is a pretty good point to consider it equal. You can’t express arbitrary javascript programs in JSON, or any other C-family languages code in data-literal syntax. The lisp family however uses the same syntax for representing data as it does for representing its program.

                                                                                        1. 5

                                                                                          Lisp just has a closer correspondence between tree and string, which makes it cognitively easier

                                                                                          Maybe not just cognitively but also in terms of programming. Many languages have complicated parsing and contexts where transformations aren’t as simple as an operation on a tree with regularity in its syntax and semantics.

                                                                                          1. 2

                                                                                            Right. Von Neumann machine code is homoiconic, but I don’t think it exhibits many of the purported advantages of Lisp?

                                                                                            1. 2

                                                                                              The one’s Ive seen are definitely harder than LISP’s to work. Now, might be different if we’re talking a Scheme CPU, esp if microcoded or PALcoded. With SHard, it can even be built in Scheme. :)

                                                                                      1. 1

                                                                                        Last step: use tools to benchmark. There are various speed test websites. They can give you additional tips. Maybe you can compress an image better for example or you forgot gzip compression.

                                                                                        1. 4

                                                                                          Should have put a Spectre-Exploit on the site and count how many vulnerable visitors he got for extra lulz.

                                                                                          1. 2

                                                                                            Open the fake one to get hit by the real one. That’s a nice idea.

                                                                                          1. 2

                                                                                            In what other languages would it be possible?

                                                                                            I guess everything with properties (functions disguised as fields) so D, C#, etc.

                                                                                            Afaik not with C, C++, or Java.

                                                                                            1. 26
                                                                                              #define a (++i)
                                                                                              int i = 0;
                                                                                              
                                                                                              if (a == 1 && a == 2 && a == 3)
                                                                                                  ....
                                                                                              
                                                                                              1. 1

                                                                                                Isn’t that undefined behavior? Or is && a sequence point?

                                                                                                1. 3

                                                                                                  && and || are sequence points. The right expression may never happen depending on the result of the left, so it would make things interesting if they weren’t.

                                                                                              2. 10

                                                                                                This is very easy to do in C++.

                                                                                                1. 5

                                                                                                  You can also do it with Haskell.

                                                                                                  1. 3

                                                                                                    Doable with Java (override the equals method), and as an extension, with Clojure too:

                                                                                                    (deftype Anything []
                                                                                                      Object
                                                                                                      (equals [a b] true))
                                                                                                    
                                                                                                    (let [a (Anything.)]
                                                                                                      (when (and (= a 1) (= a 2) (= a 3))
                                                                                                        (println "Hello world!")))
                                                                                                    

                                                                                                    Try it!

                                                                                                    Or, inspired by @zge above:

                                                                                                    (let [== (fn [& _] true)
                                                                                                          a 1]
                                                                                                      (and (== a 1) (== a 2) (== a 3)))
                                                                                                    
                                                                                                    1. 3

                                                                                                      Sort of. In Java, == doesn’t call the equals method, it just does a comparison for identity. So

                                                                                                       a.equals(1) && a.equals(2) && a.equals(3); 
                                                                                                      

                                                                                                      can be true, but never

                                                                                                       a == 1 && a == 2 && a == 3;
                                                                                                      
                                                                                                    2. 3

                                                                                                      perl can do it very simply

                                                                                                      my $i = 0;
                                                                                                      sub a {
                                                                                                      	return ++$i;
                                                                                                      }
                                                                                                      
                                                                                                      if (a == 1 && a == 2 && a == 3) {
                                                                                                      	print("true\n");
                                                                                                      }
                                                                                                      
                                                                                                      1. 2

                                                                                                        Here is a C# version.

                                                                                                        using System;
                                                                                                        
                                                                                                        namespace ContrivedExample
                                                                                                        {
                                                                                                            public sealed class Miscreant
                                                                                                            {
                                                                                                                public static implicit operator Miscreant(int i) => new Miscreant();
                                                                                                        
                                                                                                                public static bool operator ==(Miscreant left, Miscreant right) => true;
                                                                                                        
                                                                                                                public static bool operator !=(Miscreant left, Miscreant right) => false;
                                                                                                            }
                                                                                                        
                                                                                                            internal static class Program
                                                                                                            {
                                                                                                                private static void Main(string[] args)
                                                                                                                {
                                                                                                                    var a = new Miscreant();
                                                                                                                    bool broken = a == 1 && a == 2 && a == 3;
                                                                                                                    Console.WriteLine(broken);
                                                                                                                }
                                                                                                            }
                                                                                                        }
                                                                                                        
                                                                                                        1. 2

                                                                                                          One of the ‘tricks’ where all a’s are different Unicode characters is possible with Python and Ruby. Probably in Golang too.

                                                                                                          1. 7

                                                                                                            In python, you can simply create class with __eq__ method and do whatever you want.

                                                                                                            1. 4

                                                                                                              Likewise in ruby, trivial to implement

                                                                                                              a = Class.new do
                                                                                                                def ==(*)
                                                                                                                  true
                                                                                                                end
                                                                                                              end.new
                                                                                                              
                                                                                                              a == 1 # => true
                                                                                                              a == 2 # => true
                                                                                                              a == 3 # => true
                                                                                                              
                                                                                                          2. 2

                                                                                                            In Scheme you could either take the lazy route and do (note the invariance of the order or ammount of the operations):

                                                                                                            (let ((= (lambda (a b) #t))
                                                                                                                   (a 1))
                                                                                                              (if (or (= 1 a) (= 2 a) (= 3 a))
                                                                                                                  "take that Aristotle!"))
                                                                                                            

                                                                                                            Or be more creative, and say

                                                                                                            (let ((= (lambda (x _) (or (map (lambda (n) (= x n)) '(1 2 3)))))
                                                                                                                    (a 1))
                                                                                                                (if (or (= 1 a) (= 2 a) (= 3 a))
                                                                                                                    "take that Aristotle!"))
                                                                                                            

                                                                                                            if you would want = to only mean “is equal to one, two or three”, instead of everything is “everything is equal”, of course only within this let block. The same could also be done with eq?, obviously.

                                                                                                            1. 1

                                                                                                              Here is a Swift version that uses side effects in the definition of the == operator.

                                                                                                              import Foundation
                                                                                                              
                                                                                                              internal final class Miscreant {
                                                                                                                  private var value = 0
                                                                                                                  public static func ==(lhs: Miscreant, rhs: Int) -> Bool {
                                                                                                                      lhs.value += 1
                                                                                                                      return lhs.value == rhs
                                                                                                                  }
                                                                                                              }
                                                                                                              
                                                                                                              let a = Miscreant()
                                                                                                              print(a == 1 && a == 2 && a == 3)
                                                                                                              
                                                                                                            1. 14

                                                                                                              As someone whose users are almost exclusively in emerging markets and remote and underserved locations. This doesn’t even touch the surface. I’ve been in some places with a proper broadband connection in an enterprise environment and been completely unable to load lots of common web sites.

                                                                                                              Not only that, we’re constantly developing for whatever the latest version of android and iOS, windows. A lot of the world is just getting away from Android 4.4 to be honest.

                                                                                                              Just the same with laptops. There are a lot of 1024x768 laptops roaming around out there still, users who skip big updates because they just plain take too long to download or are impossible to download on a flaky or expensive connection. When you see those charts showing the breakdown of the market of how many people are using a specific version of an OS. A certain chunk of those people who are still running Old, insecure, feature-lacking versions that you give a chuckle or tsk tsk to, literally can’t upgrade.

                                                                                                              People are climbing to the tops of hills near their villages to try and get enough signal to make a phone call still, we have situations where someone has to travel around weekly from town to town to sync data from users phones because the cell coverage and data have been too flaky.

                                                                                                              Another thing to remember is that you might be costing these people real money. You will not find cheap 2gb monthly data plans in some of these places. A lot of people are using the cheapest top up cards. So when your web site eats through their data plan by serving umpteen js files and making a billion calls to external services, serving bloated Ajax payloads, tracking calls, etc… just to load your home page, you’re actually using up a finite resource for a user as they may have to travel to another town or wait ages to be able to afford another top up.

                                                                                                              This scales out though to all aspects of a digitally connected existence. Overly designed emails loading external image assets. That png image that you never compressed properly because hey, 1mb images aren’t that bad on your fibre connection. Compression ratios for video streams, ads that stream audio on page load, ads themselves, persistent connections constantly passing data back and forth, etc…

                                                                                                              Pages that fall apart when one of the assets times out downloading though the page is sitting there on screen.

                                                                                                              I’m not saying that we should strip all of the good stuff we’ve managed to create, or that we should go back to the geo cities ages. Or even that we should always build for the least least least common denominator all the time, but if you’re a content producer or web app or desktop/mobile app that wants to be usable by everyone, everywhere you need to exercise a bit of common sense. If you’ve got a successful app that’s failing to pick up in some key target locales that you wanted to serve, or you have no traffic from for instance countries in the African continent then it might be worth investigating what your product operates like in those regions.

                                                                                                              The other half of this is that our innate ability to understand the internet and how it works isn’t the norm for everyone. In the west and developed countries, we have an almost innate immediate understanding of how a piece of software or a web site works from years of exposure to software and technology. We know how a button should look, what a select field does, how urls work.

                                                                                                              For some populations, even email is still a foreign concept. I had a user who just assumed that his first and last name @gmail.com would belong to him and he could access it because that’s how email works right?

                                                                                                              Or had users who fill out a form and show legitimate fear about pushing the submit or save button on screen because it feels very final and they don’t know what will happen when they do it. It’s a scary door and you don’t know what’s on the other side. It’s all still very much magic.

                                                                                                              We have so many people who don’t have email addresses themselves that we started just making up email addresses on our domain (not real accounts). A lot of users put it in as their login having no idea why that funny @ symbol is in their username.

                                                                                                              Sorry, rant over. Just this is something that over the years has been a constant driver in the back of my head and informs a lot of my design and development patterns. And it’s more complicated than just needing to minify and gzip your stuff.

                                                                                                              1. 3

                                                                                                                Problem is it’s really really hard to develop software to solve problems you didn’t even know existed. I’d love to make software that works for everyone but I don’t know how everyone uses their devices. I don’t know how a colorblind person sees or how a person in africa understands how to use email. And I certainly don’t have the money to visit all these people and ask them. The best I can do is build it to cover all the use cases I understand and people can send me an email or open a bug report if I missed something thats really important to them.

                                                                                                                  1. 3

                                                                                                                    I’m not saying that you should build your software to meet every conceivable combination of personal IT experience, connectivity requirements, etc… far from it as that’s pretty much not feasible even for a lot of large teams. In my case, my target users are these groups specifically and I’ve seen the frustration first-hand where people have been asked for instance to install an app on their phone and because all the person has is an android 4.4 phone, the app doesn’t exist for them. Or we’ve had a user call up because their data plan is depleted and the only thing they did was browse some news sites and blogs so we have to top up their sim cards constantly.

                                                                                                                    I’m just saying, if you’ve got an app, application, web app, mobile app or whatever, and you look and see that you have a high rate of abandonment, or no installs whatsoever in a region and you’d like to have uptake in that region, you need to think about how your product would operate under those circumstances. Even large-scale enterprises are guilty of this from what I’ve seen, so it’s not just a young developer not knowing any better situation. I’ve had big corps pitch their well-developed software solutions for some of our situations and their applications fall flat on their face in the meeting because they just couldn’t deal with the network health in our locations.

                                                                                                                    If you’re happy to never address the segment of the market then don’t worry about it, but if you want a bigger piece of the pie, you don’t have to travel to Africa to identify potential problems, turn on throttling in chrome dev tools and throttle down to a 3g or lesser connection, Use a connection throttler to test how your mobile app or desktop acts when you run it. Rename your largest JS dependency to something else and see what happens to your page when you try to load it (basic simulate a timeout on a resource). Look at chromes network tab and tally up how much data actually comes into your app when you load the page. Simple stuff, basic common sense stuff.

                                                                                                                    There was another article on here a few days ago about someone who provides a service that allows people to use WhatsApp on lower-end phones through a browser and people went nuts for it. If you can work around it, you can achieve some great growth.

                                                                                                                    And I don’t just mean the African continent either, there are plenty of people in rural UK, America, Canada, South America, Europe and the Pacific that are surviving on 3g modems, tethering their cell phones, etc… I can see a major town from my house, but I can’t get any better than a dial-up connection speed on my broadband and some days I have to resort to using a 3g modem to get any kind of internet at all, I have to walk out of the house and up the road a bit to get a good cell phone signal. I can literally pull out my driveway and people have fiber to their property. There are swathes of developed countries populations which don’t have access like you would in a metropolitan environment still.

                                                                                                                    If addressing these people isn’t a concern, then don’t worry about it, well, maybe have a little common sense with using compression, clear out deprecated and unnecessary code, style assets, etc… (which does go a long way) and carry on with your life. But if you want to grow globally outside of your current market, you’ll need to adjust your thinking a little bit, set some constraints on how big you’ll allow your application to be, don’t import an entire date library just to format dates, give yourself a delay time before adopting newer things like flex-box, etc…

                                                                                                                    1. 7

                                                                                                                      I remember a story from Youtube: They saw bad loading times in their stats and went on to optimize that by making Youtube more lightweight. After the optimizations the stats got even worse! However, the usage went up. A lot more people, for whom Youtube timed out, were now able to use it.

                                                                                                                      Unfortunately, Google cannot find me the original story. I believe it was some Google blog.

                                                                                                                      1. 1

                                                                                                                        Even a little bit like that goes a long way, long wait times are basically standard on loading a page for some people, so when a page does actually load it’s great.

                                                                                                                        A weird instance we had once was a user that was using a 3g dongle for their data, and they were requesting a long-running process in one of the earlier versions of our system, the process would run, and they would sit there waiting for it to complete. For this user (and a few similar cases until we changed how the long-running process was called) their view of it was that it was still running hours later even though the job had finished ages before. Took forever to figure out that over his 3g dongle, the connection was getting swapped on the telecom side, when the app responded with the result, the user wasn’t there anymore even though they never lost their actual connection to the internet. The problem was there was nothing in between the user and our servers to report to the user that the request had failed because of a weird time out implemented on the telecoms side. Swapping to a hardline or office wifi and the process completed and worked fine.

                                                                                                                1. 13

                                                                                                                  SQLite is much more practical than people give it credit for. Your application probably doesn’t need PostgreSQL. (Although PostgreSQL does have a lot of features that make it a nicer choice.)

                                                                                                                  1. 2

                                                                                                                    I’m a fan as well. My colleagues switch to PostgreSQL now, because SQLite has limitations with ALTER TABLE. The Play framework uses database evolutions written in SQL so that is easier with PostgreSQL.

                                                                                                                  1. 7

                                                                                                                    First of all, I love love love how vibrant the Lobsters formal methods community is getting. I’m much more likely to find cool FM stuff here than any other aggregator, and it’s awesome.

                                                                                                                    Second, maybe I’ve been spending too much time staring at specifications, but I’m not seeing how level 1 is different from level 2. Is level 1 “this is broken for probable inputs”, while level 2 is “this is broken for some inputs”? Different in degrees of probability.

                                                                                                                    1. 4

                                                                                                                      Level 1 is statements about specific executions; level 2 is statements about the implementation. This is for all statements, not just correctness.

                                                                                                                      1. 1

                                                                                                                        First of all, I love love love how vibrant the Lobsters formal methods community is getting.

                                                                                                                        Me too.

                                                                                                                        I believe crustaceans care more for high-quality code than the average programmer. Maybe pride of the craftsmen? This is a distinction to Hacker News, where monetary considerations get more attention. Formal methods is certainly one of the big topics to improve the quality of code.

                                                                                                                        1. 1

                                                                                                                          I also love that we get more formal methods discussions taking place here!

                                                                                                                          I am however not sure how much “depth” is accepted; should I post any paper I find interesting here, with short summaries why and personal reflections?

                                                                                                                          1. 2

                                                                                                                            I usually just post the papers. One thing I do, though, is try to make sure they show some practical use along with what they could or couldn’t handle. Especially in abstract so readers can assess with a glance. I’ll sometimes add summaries, reflections, brainstorming, etc if I feel it’s useful. I will warn the acceptance are hit or miss with many of the PDF’s getting 1-3 votes. Then, out of nowhere, they really appreciate one. Also, I only submit anything important on Monday-Friday since many Lobsters seem to be off-site on weekends.

                                                                                                                            So, that’s been my MO. Hope that helps.

                                                                                                                        1. 9

                                                                                                                          Counterpoints: Correctness is Easily Defined (maybe), Levels Mix Up, and People Issues Over Tech Issues

                                                                                                                          (or “Brace Yourself Peer Review is Coming!”)

                                                                                                                          My typical model for describing correctness is Abstract, State Machine since ASM’s are more like Turing machines for structures. Really easy to learn for ordinary programmers compared to most formal models. Moore and Mealy state machines are basic for this on finite side. If modeling as such, then correctness is simply that your system produces the intended behavior in terms of external output and/or internal states/transitions upon given input(s). Yours are subsets of that at best. Definition 1 is subset observing one or more state machines give incorrect output for input. Definition 2 is subset identifying specific inputs and transitions that led to incorrect output. Definition 3 kind of jumps sideways to focus on arguments made about specifications rather than the specifications themselves. Both might be incorrect. Correctness still reduces to same thing that I could model each of your definitions with. Even modifying it to say “correct under reasonable operating assumptions” is same where “CPU, RAM, or HD aren’t faulty” becomes another input that must be true when CurrentState or Output are correct.

                                                                                                                          In objective case, your argument that correctness is hard to define or has many definitions seems wrong. If applied to subjective definitions, it’s true where people come up with definitions narrow enough to miss important aspects of correctness. You do a good job illustrating several perspectives such as runtime, code, and logical analysis that can help or hurt depending on what aspect of correctness one wants to achieve. These levels might be too simplistic in real world to tech correctness, though. Some examples follow.

                                                                                                                          In Level 2: Code, we do consider some behaviors that can not happen directly in code when writing reliable or secure programs. We might write code to get the algorithm correct first, then do an assessment of known issues outside of code that affect code, modify the code to include those solutions, and then we’re done. It still happens at code level even if originally the knowledge came from a different level. A good example is software caches that aren’t really about the semantics of the code but we do modify the code we write to reduce misses. It’s a requirement/design detail that, when applicable to code, causes a transformation of that code. Eventually, the coder might automatically write code that way as part of coding style (esp with caches). Other examples include running out of memory at random points, bit flips, or side channels in CPU’s. These have code styles to mitigate them people can use without even understanding the logic behind it which can be enforce by code analysis tools.

                                                                                                                          In Level 3: Design/Logic, it assumes logic specs and implementation are a different thing. The most successful by the numbers use of formal methods in imperative programs are contracts and assertions. Most practical that subsets formal methods being Design-by-Contract. These tightly integrate logic dictating correctness with the source code where they’re both created and considered at Level 2. In some implementations, the assertions become runtime checks that are activated in Level 1. So, the assertion-oriented methods operate at Levels 2-3 simultanteously when writing specs/code with 1-3 happening together when testing specs/code.

                                                                                                                          “That can only be seen when doing formal verification, where a program’s properties and assumptions are written just as concretely as its source code. “

                                                                                                                          Or Design-by-Contract with verification condition generator. The more specs and VC’s they see, the more complex the behavior might be. These things could possibly be converted to English statements about the software, too. There’s ongoing work in natural-language provers. You’re right that the common case is the specs are implicit in code or in developers’ heads “scattered” about in inconsistent way. That’s a strong argument for explicit specs.

                                                                                                                          “Level 3 is all about how modular are the interactions between the components of your software”

                                                                                                                          They’re going to tell you OOP or FP plus dynamic languages solve that. It’s why Smalltalk and LISP systems are so malleable. They also supported live debugging and updates. Many long-running systems with acceptable level of failures achieved with English specs plus memory-safe, dynamic, maintainable code. Not a formal spec in sight. Lots of half-assed versions of aforementioned concept in mainstream languages with bolted-on abstractions, middleware, clusters, and so on. You might want to think on this more to determine how to describe why you’re solution is superior to that for practical reasons. If it even is given that static, verified code vs safe, easy-to-modify, dynamic code being better in general case for “acceptable correctness” is ongoing debate among developers in CompSci, industry, etc.

                                                                                                                          “At Level 2, this was totally fine, since freed memory in DOS was valid until the next malloc, and so the program worked. At Level 3, this was a defect, “

                                                                                                                          Another reason I don’t like the levels for these subjects is we already have the SDLC models and other stuff like that which has a way to specify requirements, design, and code with traceability. There’s piles of methodologies and tools for doing this. What this section says is that, ignoring best practice, a developer only considers the code rather than the hardware or OS considerations that will be introduced in “requirements” or “design” documents saying the code has to fit within that model. Then, you illustrate the problem that ignoring everything but code creates. I know there’s plenty of developers out there doing it but it doesn’t mean we need new levels or other models for the basics. Just illustrate with good examples like yours why we have those basic concepts there for software development lifecycle. Note that I’m not pushing order of Waterfall so much as the categories of development.

                                                                                                                          “The HTTP “referer” is forever misspelled, and that SimCity special-casing code is still there 30 years later.”

                                                                                                                          You’re treating it all as one thing. The SimCity software might have been cheap to the people who built it. The company still makes games. So, I’m guessing SimCity made them a lot of money, too. That software was totally successful as a product at its goal “for that group at that point in time.” It’s true the software might become a problem for another group down the line needing it to do a different thing due to technical concerns in the code. However, it was at acceptable quality and successful before with it not being in another context later. What you’re seeing here is the effect of economic/human factors dominating technical ones when deciding if specific methods are good methods (esp good enough).

                                                                                                                          We should always consider such things in our exhortations because those listening in FOSS or industry sure as hell will. Many’s model of operation means they don’t need to care about long-term quality. FOSS folks or CompSci students (esp w/ non-reproducible research) might not need to care about even current quality. Others don’t need high runtime quality so much as “it works often enough with it easy to fix.” Others add fast rate of change to that. Others, esp bigger companies, will talk developer costs for specific lines of code on top of that. The goals and metrics vary across groups. If you want to convince them, you need to target your message to those goals and metrics. If you don’t want to relent, you’ll have to target groups that are using your goals and metrics such as low-defect rate, higher flexibility for change, increased predictability (aka lower liability), and so on. That’s a smaller segment.

                                                                                                                          Like you, I spent a lot of time talking about objective metrics focused on the tech when the groups’ intended goals, failure tolerances, and subjective metrics are where the ability to influence is at. I wasted a lot of time. Now, I focus on teaching them how defects can be found faster with little extra work (esp at interfaces), how software can be changed faster, how it can be diagnosed faster, how it can be fixed faster, and how rollback can work well and/or fast. And with what cost, level of support, etc. This is their language.

                                                                                                                          “I now have two years experience teaching engineers a better understanding of how to avoid complexity, improve encapsulation, and make code future-proof.”

                                                                                                                          Glad you’ve been doing that. Keep at it. All I’m doing here is showing you how to improve the message.

                                                                                                                          Part Two

                                                                                                                          “ The three levels deal with different views of a program: executions, code, and specifications. Each corresponds to its own kind of reasoning.1 Let’s bring in the math!”

                                                                                                                          Now this is true and where you’r use of levels shines with good examples like overflow. However, the analysis seems to miss that these are common problems at the code level where simple checks can take care of them. Even your malloc example might be handled in a library implementation where some things the spec tracks were tracked by compile-time analyses or run-time library. Such non-formal methods ares typical of memory-safe languages, embedded systems catching integer overflows in code, watchdog timers to dodge termination proofs, and so on. The overhead they have is acceptable to many or maybe most given dominance of such methods in industry and FOSS. Others who want less overhead or more determinism might like the techniques your bring to the table for Level 3. I already refuted separating them too much with examples such as Design-by-Contract that can do all of it at once with lots of automation (eg property-based testing or fuzzing plus runtime checks). I agree prior work shows many tools work best focusing on just one level, though.

                                                                                                                          I think you prefer clean, isolated, and static ways of analyzing or looking at system correctness when real world is just messier in terms of solutions available for one or multiple levels. It’s a lot messier. Embrace the mess! Like a consultant, factor it into your analyses so your recommendations show the various options at each level for different classes of problem with the trade-offs they take: time/money saved now or later via less debugging or fixes earlier in SDLC, extra time/money for specific verifications (these two biggest where development pace is biggest), compile-time vs runtime analysis, needs runtime checks or not at what cost, and support in their tooling of choice. It’s more work for people like us to do this. However, as some see value and more uptake happens, network effects might kick in where they do some of the evangelizing and tool work for us. That’s on top of potential niche market for specific solutions that work well like we see in DO-178B/C for static analyzers and so on.

                                                                                                                          1. 11

                                                                                                                            You should make a blog or something and keep all the longer form writing there. Then you could reference it easily on forums.

                                                                                                                            1. 11

                                                                                                                              I’m actually closer to having a blog than before. It’s probably going to be up this year after a decade of procrastination on that.

                                                                                                                              1. 2

                                                                                                                                That would be a blog, where I would subscribe before you even publish something. Please do. :)

                                                                                                                                1. 1

                                                                                                                                  What has been your primary hindrance, if you don’t mind my asking? There are (and have been) many free blog hosting platforms, that make it quite trivial to start writing. Also, if you desired to host your own, services like github pages and netlify offer very compelling free tiers.

                                                                                                                                  1. 1

                                                                                                                                    There are (and have been) many free blog hosting platforms

                                                                                                                                    Is there a good one?

                                                                                                                                    1. 1

                                                                                                                                      I dont mind you asking but rather not say. You could just say my background, current circumstances, and procrastination habit all added up to hold off since I already had places to share ideas. On that latter point, I also found it easier to help others by going where crowds were already at than trying to do lots of self-promotion. I was drafting the heavy hitters instead of building a turbo-charged, 18 wheeler of my own. Whether good or bad idea who knows.

                                                                                                                                      I am switching gears on quite a few things this year. Gonna try a piece or just a few at a time to avoid overwhelming myself.

                                                                                                                                2. 3

                                                                                                                                  Hi, OP here.

                                                                                                                                  Thanks for the long response. The typography of this comments section is really not meant for text of this length. As such, I’m having a lot of trouble reading this, for multiple reasons. I can’t tell how you disagree with me. In fact, I can’t even figure out if you disagree with me.

                                                                                                                                  1. 6

                                                                                                                                    Okay, I’ve read it through again. Here’s my attempt at a summary of what you wrote:

                                                                                                                                    • Automata are a good way of writing specifications. Levels 1 and 2 have analogues in the automata-based style of specification.
                                                                                                                                    • Something about me saying that correctness has many definitions (which is only kind of true)
                                                                                                                                    • Changes to code are informed by the design
                                                                                                                                    • Partial specifications may be embedded in the code via assertions
                                                                                                                                    • Something about OOP. Who is the “they” that’s telling me it’s going to solve modularity? Is this responding to something I said? (Sidenote: Objects are very misunderstood and underrated among the intelligentsia. I highly recommend reading William Cook.) Also, something about “my solution.” I don’t know what “my solution” is supposed to be; I’m just providing a nomenclature.
                                                                                                                                    • Something about how developers hopefully don’t think about nothing but the code when programming.
                                                                                                                                    • For some kinds of software, most of the value to the producer occurs in a short time horizon.
                                                                                                                                    • Different kinds of software have different value/time curves.
                                                                                                                                    • Bug catching tools exist.
                                                                                                                                    • The real world is messy.

                                                                                                                                    None of this seems at all opposed to anything I wrote. Am I missing a key point?

                                                                                                                                    There is one minor point of difference though: I’m not sure that Level 3 has an analogue in some variations of automata-based specification. This is because an automata-based specification is a very whole-system view of the world. There’s not exactly a notion of “This implementation automaton is a correct refinement of that specification automaton, but for the wrong reasons” unless you have a hierarchical construction of both. Hence, if you try to think of your system as implementing some kind of transition system, you may not realize if you have poorly-defined component boundaries.

                                                                                                                                    I expect this phenomenon is related to why LTL-based program synthesis performs so poorly.

                                                                                                                                    1. 4

                                                                                                                                      Sidenote: Objects are very misunderstood and underrated among the intelligentsia. I highly recommend reading William Cook.

                                                                                                                                      Sidenote to your sidenote: while we’ve mostly accepted that there’s several divergent strands of Functional Programming (ML, Haskell, Lisp, APL), we haven’t really done the same with OOP. That’s why people commonly conflate OOP with either Java ObjectFactoryFactory or say “that’s really not OOP, look at Smalltalk.” While Java is a Frankenstein of different strands, Smalltalk isn’t the only one: I’d argue Simula/Eiffel had a very different take on OOP, and to my understanding so did CLU. But a lot of CLU’s ideas have gone mainstream and Eiffel’s ideas have gone niche, so we’re left with C++ and Smalltalk as the main ‘types’ of OOP.

                                                                                                                                      1. 5

                                                                                                                                        There are old programming concepts called objects, but they don’t necessarily correspond to meaningful constructs. Cook and Cardelli did discover something that does capture what’s unique about several strands of OOP, and also showed why CLU’s ideas are actually something else. I consider this a strong reason to consider their work to be the defining work on what is an “object.”

                                                                                                                                        This is largely summarized in this essay, one of my all-time favorites: http://www.cs.utexas.edu/~wcook/Drafts/2009/essay.pdf

                                                                                                                                        1. 1

                                                                                                                                          By the numbers, I’d consider it whatever C++, Java, and C# did since they had the most OOP programmers in industry. Then, there’s the scripting languages adding their take. So, my hypothesis about mainstream definition is that what most people think OOP is will be in one of those camps or a blend of them.

                                                                                                                                      2. 3

                                                                                                                                        I’ll take the main comment paragraph by paragraph showing what I did for context:

                                                                                                                                        1. I disagreed with your three models for correctness in favor of one that says “Do states, transitions, and outputs happen as intended on specific inputs?” All yours fit into that. There’s also formal models for it deployed in high-assurance CompSci and industry.

                                                                                                                                        2. I agreed your levels represent different vantage points people might use. I also agreed some tools work best focused only on them.

                                                                                                                                        3. I disagreed you need these to explain why code alone isn’t sufficient. The standard software, development lifecycle (even Waterfall) shows places where problems show up. Lots of articles online to pull examples from with your DOS thing being one. Best to just use what’s widely-used and well-understood for that.

                                                                                                                                        4. I disagreed that stuff in the three levels stayed separate during development. I gave specific examples of where stuff at one was solved at another without mentally leaving that other level. I also showed methods that do multiple ones simultaneously with strong, code focus.

                                                                                                                                        5. I disagreed that strong correctness of software (a) mattered for many groups or (b) mattered in long term. You have to examine the priorities of people behind products or projects to know if it matters or how much. Then, tie your recommendations to their priorities instead of yours. Ours if it’s software quality improving. :)

                                                                                                                                        6. I disagreed that strong, logical methods were needed for long-term when specific examples from Smalltalk/LISP to industry’s knock-offs regularly produce and maintain software that meet organization’s goals. Since they’re status quo, you must show both those kind of solutions and yours side-by-side arguing why your costlier or stranger (to them) methods are better.

                                                                                                                                        7. I agreed some would have your view in niche segments of FOSS and industry that are willing to sacrifice some upfront-cost and time-to-market to increase quality for higher levels of (insert good qualities here). You will do best marketing to them again tying the message to their interests and metrics. I gave DO-178B (now DO-178C) market as example where they invest in better tooling or buy better-made software due to regulations. Ada/SPARK, Astree Analyzer, formal methods, and so on have more uptake in safety-critical embedded than anywhere else except maybe smartcards.

                                                                                                                                        So there’s a summary that will hopefully be more clear by itself or if you read the original post with it in mind.

                                                                                                                                        1. 5

                                                                                                                                          Thanks. That I can read.

                                                                                                                                          First, please tell me where I say that strong correctness is important or that we should be using heavyweight formal methods in today’s industry. I don’t believe either of those statements except in a small number of high-assurance cases, but you’re not the first to think that I do. Also, what is the “your solution” that you speak of? Points 5-7 seem to all be arguing against something I don’t believe. (As an aside, where are you getting your claims about tool adoption? My understanding from talking to their competitors is that Astree has no market share.)

                                                                                                                                          Point 4: Obviously, you need to think about code (level 3), write code (level 2), and run code (level 1) concurrently with each other. Thinking about these three levels of abstraction absolutely do happen concurrently. What should set off alarm bells is writing code that special-cases a certain input (level 1 influencing level 2), and shaping a design around a poorly-thought-out implementation (level 2 influencing level 3). Obviously, your value/time curve will determine when exceptions are worth making.

                                                                                                                                          Point 3: Ummm…..I don’t know what to say. Obviously, there are a lot of people with no training in formal methods who are nonetheless very good software designers. I do believe that the concepts of PL and formal methods, in various incarnations, whether logic-based or automata-based, are the correct way to think about many software design concepts that were previously only vaguely-defined. These all provide a notion there is a layer of reasoning which shapes the code, but is invisible when looking only at the code. This is what I call Level 3, or “the hidden layer of logic,” and closely related to what Don Batory calls “Dark Knowledge.” (Someone on HNH just linked me to a Peter Naur article which might discuss the same concept.) Probably the clearest example of this concept is ghost code, which many static analysis tools rely on. I also believe that understanding the hidden layer of logic is a shortcut to the higher levels of software design skill, as I’m proving in my coaching practice.

                                                                                                                                          1. 3

                                                                                                                                            I could’ve misread the intent of your post. I thought you were promoting that people consider, maybe adopt, specific ideas. If you’re just brainstorming, then you could read me as liking or shooting down some of the brainstormed ideas. It all depends on what your goal is. I’ll answer some of this.

                                                                                                                                            First, one thing about your post that looks like an evangelistic push of specific solutions like applying your concepts of Level 1-3 or software engineering practices in general. Here’s some examples of why I thought that:

                                                                                                                                            “No, our goal is to be able to continue to deliver working software far into the future.”

                                                                                                                                            “People sometimes tell me how software is easy and you can just… Hogwash… This is why it’s important to get programs right at Level 3, even if it passes all your testing and meets external requirements.”

                                                                                                                                            “This is why it’s important to make your APIs conform as strictly to the spec as possible”

                                                                                                                                            “the most important parts of our craft deal not with code, but with the logic underneath. When you learn to see the reasoning behind your system as plainly as the code, then you have achieved software enlightenment.”

                                                                                                                                            “always think of the components of your… So much becomes clearer when you do, for logic is the language of software design.”

                                                                                                                                            Your replies to me were talking like it’s idle categorization or brainstorming but those quotes sound like pushing specific practices to some audience of programmers to solve their problems. I assume industrial and FOSS programmers as default audience since they account for most working, practical software out there. If you’re (a) talking to real-world programmers and (b) pushing specific advice, then I responded to that: in terms of what their requirements are; what methods they use (eg design review, OOP, patches, runtime checks); told you to compare what you were pushing to good versions of what they did; and, if your methods still sound better for them, then modify your pushes to argue stuff more meaningful to them, esp project managers or FOSS volunteers. That what I meant by your “solutions” or “recommendations” with my counterpoints on suggested modifications.

                                                                                                                                            I should’ve added that this quote that was 100% in agreement with my recommendations:

                                                                                                                                            “I always recommend trying to think in pure concepts, and then translate that into the programming language, in the same way that database designers write ER diagrams before translating them into tables. So whether you’re discussing coupling or security, always think of the components of your software in terms of the interface, its assumptions and guarantees”

                                                                                                                                            Back to other points.

                                                                                                                                            “Obviously, you need to think about code (level 3), write code (level 2), and run code (level 1) concurrently with each other. “

                                                                                                                                            I’m glad you intended for them to know that. It would be obvious if you’re audience already knows benefits of level 1-3 activities but then you wouldn’t have to explain the basics. Problem was your audience (as I perceived them!) was composed of people you were convincing to use more than coding and a debugger. Given Design-by-Contract reactions, I know it’s not going to be obvious for many of them that logic and code can work together in a way that’s also checked at runtime. Maybe it should be in a similar, future write-up if you do them that they can interleave or that some methods do several at once.

                                                                                                                                            “These all provide a notion there is a layer of reasoning which shapes the code, but is invisible when looking only at the code.”

                                                                                                                                            It’s a neat way of describing the other factors. The industrial developers do have methods for that, though, that you might leverage to help them understand. Big attention goes to modeling languages with UML’s probably being most widespread. In safety-critical, Esterel SCADE is good example. Anyway, the industrial side has modeling techniques and diagrams for various parts of the lifecycle that show relationships, constraints, and so on. Adoption varies considerably just like with strong design in general. Many will have seen that stuff, though. Starting with something like it might increase how fast they understand what you’re saying. You can then show how formal logic has benefits of simplicity, consistency, and tooling. And I tend to especially highlight the automated analyses that can happen since resource-constrained groups love solutions that involve a button push. :)

                                                                                                                                            “I also believe that understanding the hidden layer of logic is a shortcut to the higher levels of software design skill, as I’m proving in my coaching practice.”

                                                                                                                                            Definitely! In that case, you also have people who are trying to improve rather than randomly reading your blog. It’s a different audience willing to invest time into it. It’s definitely beneficial to teach them about the implicit stuff so they see just what kind of complexity they’re dealing with. I fully support that. An example I give to people learning formal verification is this report doing a landing system in Event-B. Specifically, I tell them to note how the small number of requirement and design details balloons into all the stuff in section C onward. I tell them, “Although some is due to the logic, much of that is just you explicitly seeing the huge pile of details you have to keep track of throughout your system to make it correct in all situations. It’s always there: you just don’t see it without the formal specs and explicit over implicit modeling.” That always gets a reaction of some kind. Heck, for me, it made me want to reconsider both not using formal methods (what am I ignoring?) and using formal methods (wow, that’s a lot of work!). :)

                                                                                                                                            1. 3

                                                                                                                                              Oh boy, this is getting unwieldy. Replying in separate comments.

                                                                                                                                              1. 3

                                                                                                                                                I view this post as saying “Here’s a good way to think about software.” This does make it easier to come up with a number of design claims, such as “conforming strictly to an API reduces the complexity of future clients.” Obviously, everything has tradeoffs, and needs to be tailored to your situation. For the better engineers I’ve taught, the reaction to a good fraction of this material is “It clarified something I already had a vague intuition for,” and that is also the aim of this post.

                                                                                                                                                This lens does lead to a number of recommendations. In the follow-up post, currently #3 on Hacker News, I give a counterintuitive example of reducing coupling. This is, of course, only useful to those who desire to reduce coupling in their code.

                                                                                                                                                1. 3

                                                                                                                                                  I’m compressing most of the second half of your post into “For people already familiar with certain design-capture methodologies, you can help explain your ideas by comparing them to what they’re familiar with.” Although I once came close to taking an internship with the SEI, I can’t say I’ve had the (mis)fortune of working anywhere that used one of these methodologies (startups gotta iterate, yo), and I’ve read 0 books on UML or any of them.

                                                                                                                                                  So, if I’m going to go this route, do you have anything in mind that, say, more than 10% of Hacker News readers would likely be familiar with? Otherwise, referencing it won’t fill the desired purpose.

                                                                                                                                                  Thanks for sharing the Event-B report, BTW.

                                                                                                                                      1. 5

                                                                                                                                        I never understood why people emphasize keywords.

                                                                                                                                        Screenshot of my vi color scheme: I highlight comments and literals. Comments must be obviously distinguished from code, especially if there is code in a comment. Likewise, string literals with code in them must be obviously different. Other literals like magic numbers are an anti pattern and should be spotted easily.

                                                                                                                                        I like the idea of semantic colors. We could also highlight local variables different than global variables or fields, for example.

                                                                                                                                        1. 6

                                                                                                                                          I like the distinction of “tool” and “place”. It feels like a useful mental concept.

                                                                                                                                          I can see a relation to the economics of software development. Companies want their products to be places, so they capture a slice of your attention and deepen awareness of their brand. Nvidia is an example: It is just a single part of the computer hardware, yet it comes with a GUI tool and often demands attention.

                                                                                                                                          Free software can afford to become a tool. Imagine if you boot a Linux desktop and various involved projects show you a series of splash screens first: This desktop experience brought to you by systemd, dbus, dnsmasq, CUPS, NetworkManager, PulseAudio, Gnome, Mozilla Firefox, Gnome Keyring Daemon, gvfsd, and bash.

                                                                                                                                          1. 4

                                                                                                                                            It’s an interesting dichotomy, and drastically more interesting than the same old “minimalism” tirade I was expecting from reading the title of the post.