1. 27

  2. 8

    If you want to check out a visual language in development that’s solved all 13 of those problems, check out Enso (previously known as Luna), whose niche is general purpose data processing and visualization.

    • Difficulty with large programs: just encapsulate into named nodes to reduce toplevel clutter
    • Tedious to place each component: placing nodes can be done by just typing their names like function calls and drawing a single line through a bunch of nodes to connect them
    • Lack of formal specification: Enso is statically typed and runs on Haskell GHC
    • Need to build an editor and environment from scratch: not really, the original IDE was built on top of Atom, though it has since moved on. the compiler is built on top of GHC
    • Lack of evidence of their worth: the language’s development is funded by companies who want it
    • Difficult to debug and edit: I’d honestly prefer to edit, debug, and profile functions by seeing them as highlighted nodes rather than dig through logs
    • Lack of portability of programs: there is a textual representation, it’s as portable as your Java
    • Uses gotos and explicit transfer of control: no, it is a functional language
    • Only have global variables: no, it is a functional language
    • Has no procedural abstraction: no, it is a functional language
    • Has no parameters for the procedures: no, it is a functional language
    • Has no place for comments: gosh I don’t think this is true for any modern visual language
    • No interoperation with other languages: “Luna will let you paste any Haskell, C and JavaScript code into a component, handling the interoperability for you. (Coming soon)”

    IMO what sets it apart from other visual languages is that you edit the visual and text representations of a program at the same time, and its text representation is editable-Haskell-like rather than generated-config-file-like. Haven’t seen anything like it.

    1. 5

      And as a core differentiating aspect, Luna code has dual representation, where a purely textal form of the code vs. the graph view can be converted both ways with absolutely no loss of logic (the storage format is actually textual + (x,y) position markers). Which can be seen as an important “escape hatch” should one consider visual programming a “leaky abstraction”, letting one use all your trusted text-processing tools (most notably, diff & vanilla git) for working with their code.

    2. 7

      I spent a few years using Max/MSP/Jitter to create audio/video performances for clubs and raves right around the same time I was first learning PHP4 for work. I would never have used Max’s visual flow-based programming for “code-like” tasks but it was a brilliant way to connect data pipelines in a reactive, live control flow and hook them up to media I/O like MIDI and cameras. Visual programming has its place when inputs and outputs are well-defined and transformations on them don’t require recursion, instantiation, or deep layers of abstraction.

      1. 4

        As far as I can tell, the two by-far most successful visual “programming” examples are LabView and shader node editors.

        LabView has addressed nearly all of the issues raised in the article, and yet, in my opinion, it’s still sub-par as a programming language. I think the primary reason it has any market share at all is because National Instruments payed the marketing cost to get university engineering departments to use LabView in their undergrad labs, which meant instrument makers other than NI had to provide drivers for LabView, giving them a Microsoft-style pseudo-monopoly.

        As for shader node editors, those are a perfect example of a dedicated graphical tool being extremely useful and powerful. They don’t try to be general purpose programming languages, and that’s why they excel at what they do.

        1. 3

          Nice reference and concise summary of it!

          I also find that some of these are true not just for visual languages, but also visual programming environments.

          • Difficulty with large data or programs – Yes, reminds me of http://web.eecs.utk.edu/~azh/blog/notebookpainpoints.html
            • This post is an informal summary of our recent CHI’20 paper, “What’s Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities”
            • Not only that, but notebooks often crash with large data sets (possibly due to the notebooks running in a web browser).

          Reliability and the ability to handle large data sets is one reason I stick to the terminal, over IDEs and stuff like Jupyter.

          That said I do think the shell uses the screen poorly from a UI perspective, so it would be nice to combine the best of both worlds somehow. I guess the obvious architecture is to have the GUI in a separate process from the shell itself, so that a crash in the data display code doesn’t bring down the shell, only the “optional” visualization. Some links here: https://github.com/oilshell/oil/wiki/Interactive-Shell

          Contrarian viewpoint: I seem to see Bret Victor referenced all over the place in terms of visual programming environments. And I enjoyed his talks and a lot of what he says is insightful and true.

          However what I don’t see acknowledged is that visual programming environments are sort of “backward looking” or almost regressive. They rely on someone else to blaze a trail for you. Someone else has to envision what you want to do with the computer and provide you with a nice environment to do it.

          This is as opposed to the Unix philosophy of building your own tools. That is, language-oriented programming or DSLs are a way to get some leverage on the future. You can build up ideas that haven’t been thought before. They are more “generative”, whereas visual environments are more like a “framework” that you fit within.

          In the future there will be many different types of programming that we haven’t thought of now. Programming is more diverse than it’s ever been, and that trend will continue.

          Some examples of things on the edge: quantum computing (apparently Rigetti computing is working with Lisp-based DSLs for this), languages that allow reasoning about privacy and information flow, languages that let you write constant time crypto, etc.

          From the recent past: big data frameworks (MapReduce, spark, cloud dataflow, tensorflow) are basically programming languages, and they make visual development tools for desktop obsolete. In fact they make shells and terminals more important, and almost all IDE features less useful. In other words, they require their own separate dev tools that lag behind the language itself.

          I doubt any of the future-facing things will be addressed by visual programming environments or visual languages. In other words, visual programming has an inherently limited view of computing. It’s backward looking rather than forward looking.

          On the flip side, almost all programming is not doing something inherently new, so there’s still value to coming up with better tools. But it seems like the lag time is very long. That paper was written in 1989, and a lot of the problems still persist.

          1. 2

            I have worked with Matlab/Simulink, and I found it pretty good for control systems and state machines.

            The lack of diff tool made work with it hard, but otherwise they have their place in software development, and are pretty useful tools for some tasks.

            1. 2

              We don’t need to get rid of text as a UI for code, it works great for a great many of us. We need to get rid of text as storage mechanism and interchange format. If step zero of developing a new tool for your language wasn’t “re-parse this dead text and create a new graph that doesn’t interact with anything else” we’d see a lot more movement and progress in tooling.

              1. 1

                I’ve done a lot of programming in LabVIEW before I even started programming in a text-based language. It’s not something you can get for free, except a student version. But apart from that it has

                • Auto-Layout
                • Event-Systems
                • Compiling down to C / Embedded
                • Inspection-Mode to view where which data goes along
                • Full Cross-Plattform UI system with pretty good graphing capabilities
                • Many of C(++) functions like pointers or even ActiveX and DLL access

                Though huge single programs can get as problematic as a huge programs in single text files. I can still recommend it if you want to learn programming but you want more than what Fischertechnik or Lego has to offer. I added some of my hobby stuff on github, but they’ll always be locked behind access to a LabVIEW license, otherwise you just have some binary files in git.

                1. 2

                  I learned how to program in ROBOLAB, which I think is derived from LabVIEW. We used it to program our RCX for FIRST LEGO League. I remember making fairly sophisticated programs in ROBOLAB while still trying and failing to understand C. Textual programming languages didn’t click for me for about 5 years. I don’t know if I would currently be a software engineer if not for ROBOLAB / LabVIEW.

                2. 1

                  I’d been working with a few people on a visual game making tool for mobile (the entire tool runs on mobile! no desktop involved) + you can also post and play the games in the app. It’s still alpha and there’s no open beta yet, but you can see a video here: https://twitter.com/snikhilesh/status/1245585186135216131?s=21 It has a physics engine built-in and the editor is collaborative and you can make multiplayer games (multiplayer design may be constrained just to make sure there’s a good user experience).

                  There’s a lot of prior art out there for visual programming things, here’s a slide deck that captures a bunch: https://docs.google.com/presentation/d/1MD-CgzODFWzdpnYXr8bEgysfDmb8PDV6iCAjH5JIvaI/edit#slide=id.g1da0625f1b_0_56 (has a bunch of just general tools that include text-based envs too)

                  1. 1

                    I just wanted to add that visual languages can have formal specifications. Statecharts had quite a few attempts at specification and verification.

                    1. 1

                      I worked with Max/MSP/Jitter almost exclusively for about three years (for a stage performance system for an audiovisual artist) and used it occasionally for smaller projects and products over the past 15 years. I would say that the only points listed in the article that are not solved here in a satisfactory way are the extra screen real estate per functionality, portability and support for scoped (non-wire) variables.

                      The article doesn’t mention some other drawbacks:

                      • Less performant than compiled code, inherent to real-time nature,
                      • If commercially viable, which is a prerequisite for investing in a proper editor: not open source. I.e. no insight in or room to improve the implementation of building blocks.
                      • Less room for competition around separate subsystems or focus areas of the language/environment.

                      However, it would also be interesting to make a list with the benefits. Off the top of my head:

                      • Smaller chance of syntax errors (the most popular benefit).
                      • For some (or most?) people, visually connecting boxes is a better match with connecting ideas than putting together phrases of text.
                      • The hardest part of software engineering, dealing with complexity, takes an entirely different form; spaghetti code actually looks like spaghetti. This leads to different fundamental insights in what makes code easy to understand (this is what I personally believe to be the most interesting aspect).
                      • User-facing descriptions of black boxes are more compact than for code libraries, i.e. they look like an actual box with named in- and outlets instead of a list of available functions and commands. Apart from being more immediate to access and understand for users, this inspires authors to be sparse in the API design of their black box.
                      • There are more code annotation/implicit documentation options such as 2-dimensional ordering, encapsulation (although comparable to extracting a function) and custom code coloring.
                      • The developer of the visual language also builds the IDE so they have a stronger incentive and mandate to make all the steps between conceptualization and distribution as self-explanatory as possible.
                      • Fully proprietary, so no need to deal with tools by various vendors, no competing dialects and no incompatible library or package standards.
                      • Real-time, so no corrupt compile caches, undecipherable linking errors or other things that take time to get accustomed to but are not inherent to bringing an idea to life (this I believe is the primary reason why Max/MSP/Jitter is popular among artists and students).

                      Disclaimer: I currently work at Cycling ’74