1. 13

  2. 4

    Instead of having to think about every possible way a language might be used and account for it up front, languages with good metaprogramming facilities let the users them to fit their needs. This way the core language can stay small and focused while remaining flexible.

    While you don’t want everybody building their own language, userland extensibility can play a huge impact on the overall cleanliness of the language because new ideas can be tried out as libraries. If these ideas don’t pan out, the community moves on. Projects using these libraries will keep working, but the core language doesn’t end up accumulating cruft. I wrote about that in detail here.

    1. 2

      Is there really an “emerging idea of language-oriented programming, or LOP” as the article states? The way I see it, most modern languages are carefully crafted to balance their features (in their type and effect systems) such that I see little room for the modularity that LOP would require. I’ve never heard the term before. It sounds like we have extended the hierarchy upwards but I am not yet convinced:

      • LOP
      • Embedded DSL
      • Framework
      • Library
      • Functions, Methods
      1. 4

        I imagine people made a similar arguments against objects in the early 80s. When languages did not natively provide support for objects, it was so inconvenient that people hardly ever used them, so they necessarily felt “we have extended the hierarchy upwards but I am not yet convinced”. Now we can’t get them out of our languages even if we try. (-:

        Where languages make it — by design or by accident — easy to extend the language, language extension is rife. The paper mentions the case of JavaScript. Though JS was not invented with any meaningful metaprogramming capabilities, it left enough hooks that people have gone off and created all sorts of sub- and super-languages around it. This is also true in Ruby (“Ruby DSL” is a whole thing in itself), because Ruby also provides such hacks.

        Furthermore, a growing number of new languages have been adding macros: Scala, Julia, etc. You can view what has happened in Racket as a natural destination of where macros end up. We’re just ahead of the curve by about 20 years; there’s a good chance that as people start to use macros more in those other languages, they will slowly recapitulate all the lessons that we’ve learned, and end up creating similar solutions.

        1. 1

          When a language has an advanced type or module system, this cannot be easily extended. The language can still implement a macro system to accommodate developing patterns (like deriving RPC interfaces, support logging, or serialisation) to help cut down boilerplate code but there are limits to that. A term like LOP suggest a language can be assembled from building blocks and my lack of conviction is around that aspect.

          1. 2

            Well, no, LOP does not imply that a “language can be assembled from building blocks” — that is sometimes a consequence, but it isn’t part of the definition. The point is simply that every program has lots of small languages that are itching to surface, and languages should make it possible to do so — not in an ad hoc way, but in a way that lets those languages be turned into abstractions in their own right.

        2. 3

          It sounds like metaprogramming with DSL’s just with a new method. Language-oriented programming might be a more approachable term for that, though. If I heard it, the first things I’d think of were tools such as Rascal and Ometa that let one arbitrarily define, operate on, or use languages. That covers language part. Far as integration, a number of techs supporting DSL’s… probably a lot of LISP’s and REBOL… had a powerful language underneath one could drop into.

          So, this seems like a new instance of old ideas they’re giving a new name. I do like how they’ve integrated it into a general-purpose, GUI-based, programming environment instead of it being it’s own thing like Rascal. An old idea I had was researchers should do more experiments in building things like Rascal or Ometa alternatives in Racket leveraging what they already have to see how far one can stretch and improve it.

          1. 2

            Terra is another language in the “make DSL’s” approach to programming - although more geared towards lower level programming I think.

        3. 2

          The post talks about developing multiple languages for a single project and using them all together. Wouldn’t this mean before starting on a project you would have to learn every version used in the project and mess things up and forget something that works somewhere in the project works different in another section?

          1. 3

            Languages develop organically. You start by writing functions; you abstract these into libraries; then you start to notice patterns in the use of the libraries and try to abstract those; and eventually you notice that the abstractions don’t quite work out, and what you’d really like is a language for assembling the pieces offered by the libraries.

            What Racket does is blur the line between library and language (as the paper explains). A library can export “language constructs” just as well as it can export functions.

            For instance, consider file descriptors. You provide a library of files, including ways to open and close them. You notice that people are constantly screwing up, opening the file and forgetting to close it. You could provide a function that does both, and takes as an argument what to do in-between, but that’s unwieldy. So you can provide a constructwith-file, if you will — that lets people say what they want to do, and does the file opening and closing automatically. At some point you may even realize everyone should be using with-file only, and there’s no need to provide explicit opening- and closing-functions at all (after all, people might also close files before opening them). Voilà, you’ve gone from functions to a little language for dealing with files…seamlessly.

            1. 2

              Yes, but you get used to it the same way you learn to use a library. In a way, after you learn Racket, you lose respect for most contemporary dynamic languages…

              The only reason to use them is that they have more batteries.