1. 7
  1.  

  2. 8

    Tooling, not languages will improve software development.

    It’s 2020 and we still type characters into a static editor file. People argue about the correct placement of const in C++, where braces should go, and other minutiae about language primitives. However, directly in my editor, I can’t plug values into expressions in the code and have it show me a result–I can through a REPL in some languages, though not in the editor itself. Nor can I simulate a particular function with given parameters or set local variables in a sandbox environment to show what flow paths (simulated local code coverage) would be executed.

    Most of our code tools are still married that lines of code is the best way to present it. Our IDEs are mostly designed for code editing, not code understanding; if the code in our editors were more locally interactive, it would make code writing and reading much easier. Tools like SourceTrail and cflow are amazing steps in the right direction to understand program flow but still don’t tap into the amount of experimentation directly with code that we need.

    1. 6

      Obligatory Lisp callout. You can do basically all of this stuff for Common Lisp in SLIME. You can plug values into expressions, you can evaluate snippets of code, you can evaluate code symbolically. I don’t know if there’s a facility for fuzzing a function with N random input values then see the local code coverage, but it would be very easy to implement in Common Lisp.

      I think CL has a lot of big issues, but if we’d standardised on it 30 years ago we’d have sorted out those issues long, long ago. Instead we spend huge amounts of effort coming up with new languages when most of them could be a few macros on top of Common Lisp.

      Common Lisp’s got some great features that nobody anywhere seems to have copied. If a function signals an error, you go into the debugger, then you can do things like restart the function where the error occurred with a different value in its place. You can define custom restarts as well. The classic example is you’re parsing log entries and you signal a ‘malformed log entry’, then you can restart:

      • by skipping the log entry
      • by replacing the log entry with a given value
      • by stopping at that point in the list of log entries and moving to the next file
      • etc.

      You define these restarts at whatever spot in the code makes sense. Obviously code that skips log entries is going to be in a different place from the code that parses log entries. Because the stack isn’t unwound when you signal an error, you can restart at any point along the stack!

      Emacs’ semantic editing of s-expressions is the closest thing anyone’s ever got to semantic source code editing that actually works, too.

      1. 4

        I see replies like this often, and each time I try to imaging applying it at work, and I just can’t see it. One component I’m responsible for uses a proprietary telecom network stack (SS7) that was written about 30 years ago, and it only runs a few machines, none of which are on my desk. To even work, it needs to be connect to the phone network (and installing and managing the infrastructure is arcane to say the least—the MS-DOS command line seems friendly by comparison) so interactively trying to play with this component … yeah, ain’t never gonna happen. Just normal testing can cause the SS7 network stack to stop functioning in a mysterious way.

        I’ve given it though over the years why I dislike LISP, and I think it comes down to—everything looks the same. There’s not affordances in the code to say “this is an array reference, and this is a structure reference, and this is a function call”. And while I now realize there are more data structures than just the list, you’d almost never know it from the way it’s taught and how lists are lionized by LISP adherents (“Data is code! Code is data!”). Assembly language is similar in that it has very few affordances, and I don’t see anyone wailing over the lack of Assembly use in our industry.

        Another reason why LISP isn’t used that often, corporations don’t want it! Languages like COBOL, Java and now Go were designed because the designers didn’t trust programmers with anything powerful. Rob Pike (of Go fame) has even said as much—Google doesn’t trust their programmers to use anything “advanced” because they can’t comprehend it.

        1. 3

          The SS7 protocols are supposedly standardised, aren’t they? Is it one of those situations where every manufacturer of the equipment has their own set of proprietary extensions and incompatibilities? My first reaction to this was that I wouldn’t want to connect to production/physical/real hardware during development. I’d be wanting to connect up to some sort of test/debug device that can log and replay network traffic deterministically.

          I’ve given it though over the years why I dislike LISP, and I think it comes down to—everything looks the same.

          Everything looks the same in normal code anyway. Languages provide syntactic sugar for their standard built-in list, standard built-in hash table, standard built-in structures, standard built-in functions. But in any real code you aren’t using those things anyway. You’re using NonEmptyArray and OrderedDict and AsyncFunction etc. You don’t end up using the built-in operators and built-in literal syntax anyway, but a whole lot of function calls to do things that the builtins use syntax for.

          And while I now realize there are more data structures than just the list, you’d almost never know it from the way it’s taught and how lists are lionized by LISP adherents (“Data is code! Code is data!”). Assembly language is similar in that it has very few affordances, and I don’t see anyone wailing over the lack of Assembly use in our industry.

          That comparison doesn’t make any sense at all. It’s not true that everything in a Lisp program is a list. The code is lists. That’s only relevant when you’re writing macros or language tooling, etc. though. Code being lists doesn’t mean everything your code operates on is a list, it means that everything that operates on lists can operate on code. The distinction is really important: ‘code is data’ means ‘you can treat code like it’s really simply structured data and manipulate it easily’ not ‘everything is a list’.

          Adding things to a hash map might be written (hash-map/push m k v) in the same way you’d write HashMap::push(m, k, v) in another language. The difference between them is that to manipulate the latter in code you need a big Ast abstraction built into the standard library which is full of complexity. It’d be represented as data as something like FunctionCall(FullyQualifiedName(NameSpace("HashMap"), Name("push")), [Name("m"), Name("k"), Name("v")]). The ‘code is data’ aspect of Lisp is just that its syntax is so simple that the data representation of (hash-map/push m k v) is just… '(hash-map/push m k v). It’s really quotation that’s the important thing (along with the syntax being simple).

          I don’t think characterising my comment as ‘wailing’ is constructive or polite.

          Another reason why LISP isn’t used that often, corporations don’t want it! Languages like COBOL, Java and now Go were designed because the designers didn’t trust programmers with anything powerful. Rob Pike (of Go fame) has even said as much—Google doesn’t trust their programmers to use anything “advanced” because they can’t comprehend it.

          My friend looked up ‘where to eat in Wellington’ on Google maps and it popped up with a list of places including places marked as permanently closed. Google are totally incompetent and I don’t think we should be taking any lessons from a company that spent years telling people not to use exceptions in C++.

          Lisp is a much simpler and easier to use than C++ or Go. If you don’t trust your developers to write macros, tell them not to write macros. Institute code review rules that say ‘don’t write macros unless you’re a senior developer’.

          1. 5

            My first reaction to this was that I wouldn’t want to connect to production/physical/real hardware during development. I’d be wanting to connect up to some sort of test/debug device that can log and replay network traffic deterministically.

            I believe it was implied that those sort of test/debug devices don’t exist for his 30+ year-old infrastructure. That’s how we (I’ll throw myself in on this one, too) have to test things in/on highly-ingrained/integrated/ancient systems.

            Because a cute little test/debug device doesn’t exist for our Lovecraftian systems.

            1. 4

              Yes, SS7 is standardized, but it’s a telephony standard—you need serious money to get the actual standard, and it’s full of extensions because all of the various phone companies wanted their stuff included so it’s huge. And expensive. That’s the thing with anything telecom related—every costs money. The stack we’re using we paid six figures to license, and we’re only licensed to use it on certain systems (Solaris 5.10, and some very specific version of Red Hat Linux). And trust me, I don’t want to connect to production either, but what little we had in a lab wasn’t enough to actually simulate real life (and if pushed too hard, would fail in mysterious ways and no, it wasn’t our code but the SS7 stack—it really is alien to someone familiar with TCP/IP.

              Your bit about “in any real code you aren’t using those things” comes across as a True-Scotsman fallacy. But to some degree, yes, in C, referencing a hash looks like a function call, mainly because it is. Same thing with binary trees. But arrays have their own syntax. Structure references have their own syntax. There are some visual clues as to what the code is doing. In LISP, there’s little visual difference between referencing items from a list or an array or a hash table. And the two classes I took on LISP in college never mentioned any other data structure than the list. It wasn’t until I started looking into the language myself that I found it actually had other data structures. Perhaps the university I went to was bad, I don’t know. That’s my experience.

              And you entirely missed my point on wailing (it wasn’t you I was complaining about).

              It’s not only Rob Pike (creator of Go and Google employee) that said he couldn’t trust programmers with advanced features. James Gosling, creator of Java (and Sun Microsystems employee at the time) pretty much said the same thing about the language he designed.

              1. 2

                I don’t think we should be taking any lessons from a company that spent years telling people not to use exceptions in C++.

                Everywhere I’ve worked has expressly prohibited the use nearly everywhere of exceptions in C++ (and RTTI). The fact I have to salt my code with noexcept like a baked potato when it’s not and also the standard backed out dynamic exception specification, along with lots of newer languages not even including exceptions, indicates exceptions are looking like the spiritual successor in control flow excitement to goto.

                I don’t really have any particular issue with any LISP itself, it’s that the ease of implementation led to massive diffusion, many people haven’t been exposed to it, and its style is somewhat different from most other mainstream languages. I’ve literally showed up at work one day and had to write a LISP for an extended period and it wasn’t bad, but I understand the business folks hesitation behind introducing it (lots of pain getting up and running at the time).

                Back to my original point, I feel like there’s a lot of innovative tech from the 80s-00’s which are the code editor and code understanding version of Roman cement or Damascus steel which we haven’t rediscovered or rebuilt yet because of everyone’s focus to redo what’s already commonplace.

          2. 2

            I disagree. Languages define what tooling is possible. IDEs and non-text editors are mostly about presentation and navigation, which is very superficial. Tooling can’t do much about language semantics and expressiveness.

            Languages will continue to improve software development. Given that we’re still discussing flaws in a 40-year-old design, we still have a lot of improvements to adopt.

            1. 2

              Obligatory Smalltalk callout :-) The ST80 environment had a code browser that displayed everything In the system as a multi-paned hierarchy of classes and methods. When you drilled down into a method you got its editable source code. There were no “source files”. There was an external data file storing source code, so it didn’t have to be stored in memory, but that file was just an optimization detail.

              The good: Super easy to navigate and find code. Very focused, as you only saw the code you were working on. You never had to worry what order to put methods in, or that missing punctuation would screw up other methods.

              The bad: It was difficult to extract the code of a subsystem, or changes you’d made to something, to send it to someone else. You had to use awkward “FileOut” tools to generate text files. ST80 was very much a live system that you modified in place, not something you built from ‘source code’. This was exemplified by Dan Ingalls(?)’ quote “An operating system is the stuff you couldn’t fit in your language. There shouldn’t be one.”

            2. 6

              I’d just like to mention that the article this post is reacting to is a great read: https://eev.ee/blog/2016/12/01/lets-stop-copying-c/

              Already discussed on Lobste.rs here and here.