I’m surprised nobody has mentioned Leo. It’s a tree-based text/code editor. The tree is very flexible, and leaf nodes contain code/text/whatnot. You can even reference leafs from more than one place.
It’s not code-specific. That is, you can use it for any text-based programming language (though there will be a step to export linear-text files), or you can use it for documentation, or todo lists, or whatever. It’s very flexible.
I used it a long time back to do some literate programming. It’s evolved quite a bit since then.
Leo is maybe more an outliner than a tree editor – the nuance being that a tree editor should have a notion of schema, while an outliner can be more freeform. They’re definitely two branches of the same family tree, but I feel there are unique challenges when interactively editing structured data that needs to (eventually) conform to a given schema.
That looks quite neat; reminds me of a similar terminal program called hnb, although hnb seems more minimal and more targeted towards personal information management. Leo seems much more poweful and worth exploring, thanks for bringing it to my attention!
I find scratch.mit.edu to be somewhat good at this. There is also a paper somewhere in the vissoft2014 publications that shows using types to prevent blocks that don’t fit in a place from appearing there.
One thing to note is that often – I am trying to do an edit that is internediately not an AST, till I get to one. This is often the case when group editing many lines with a regex or such. One clearer variant of it is – intermittently doesn’t typecheck, then will. We need to be careful about which states we let the UI let the user put themselves in and which they don’t; or make it clear how they can express how much they want at each moment.
Piggy-backing on your comment, as its second part conveys more or less what I wanted to reply. I think it’s not my own intuition, that I’ve read it somewhere (probably HN): that there may actually be a reason why people seem to stick to text editors in the end. And it may be that the AST editors are too strict for expressing programmer’s thoughts, which may start on the more vague and imprecise side. For example when writing something, I may want to be able to start by “drafting” the code, to put down a sketch of my general idea, and only then run a compiler and syntax-check or even type-check it, and start molding it to fit into the rigid requirements of code. (Quite often this phase may invalidate some of my initial assumptions and force me to rethink parts of my solution, of the “thesis” expressed in code words.) In other words, I may be ignoring even syntax errors on purpose when writing the initial draft.
On the other hand, this phase could arguably be done in comment blocks / inline comments.
Interestingly to me, this idea seems to have some consequence for me when trying to use the luna-lang.org GUI. On one hand, they seem to be somewhat aware of this dynamic, as the GUI allows adding invalid nodes, and composing them together in correct way only later. On the other hand, I seem to recall that when trying to use it, I had this weird feeling that I still wanted to first sketch the overall structure of a function on a sheet of paper, before trying to “draw” it in the Luna GUI. Though this may as well have been purely because of the slowness of the GUI as of its current iteration. (Keeping my fingers crossed for the new GUI they want to release in the new year — hopefully they manage to push it out the door earlier than later.)
Also, on a more or less related note to the OP, the recent announcement of the “Tree Sitter” approach to parsing may or may not be somewhat relevant here (HN, lobste.rs).
As you point it out, the sweet spot would be for a tree editor to allow unconstrained editing (ie. no enforcing of schema/grammar) as well as constrained editing. Taking the analogy of XML, you could very well edit a tree with multiple namespaces weaved into the document, one validated by a schema, and the other one only loosely validated.
Speaking of GUIs, I think that part of the reason tree editors are still pretty much a research topic is the lack of supporting infrastructure. Programming is very text-centric and so are the tools (starting with UNIX, the shell, etc). Most GUI toolkits lack the flexibility required by interactive, user-friendly tree-editing – it’s only in the past few years that the DOM, CSS and the Web APIs have evolved to a point where this is feasible with good interactivity and reasonable performance (although, as you point out, performance is still an issue for some).
And when you think about it, it make sense: the Web is built on the notion of trees combining pieces of heterogeneous data together. With XML you have XSLT (transformations), XPath (queries) and XML-Schema (grammar). With CSS you can style the presentation and control the interaction with JavaScript.
For me, the takeaway of Tree Sitter is that we’re starting to realize that incremental transformation is the way to go for programming languages tools (parsers, compilers, etc). When you’re editing a live AST, you have the ability to stream tree deltas (patches) directly instead of re-processing the whole thing. All you need is a compiler/interpreter that is able to translate these changes into changes to the output (binary/UI).
With so many programming languages out there and LLVM becoming the de-facto substrate, I feel we’re on the right track to start rethinking syntax and free it from the shackles of a given textual representation.
That’s a good start, but I feel one of the main issue is navigation (with mouse and keyboard), and the other is dealing with eventual consistency with the schema – it would be interesting to see how you would address these in WASTE. If you’re interested in dataflow and visual languages, I’d recommend reading the corresponding chapter in Wouter van Oortmerssen’s Aardapel thesis. It features some interesting insights into the advantages and drawbacks of visual and dataflow languages.
Thanks! It’s something I’m messing with on and off as time allows, so it’s evolving very slowly.
The general idea was to force the document to always be consistent with the schema by having kind of a typeahead buffer (keystrokes don’t commit to the tree until they are parseable) and/or inserting extra leaves (you type ‘and’ on the end of an expression, it adds a temporary ‘true’ on the end to make that into a proper tree, and lets you overtype that with whatever you were about to say.
I’d like it to work with existing languages eg: Python/Javascript/Haskell because I’m more interested in the user interface aspect than the language design on this particular idea, but obviously that brings its challenges.
Aardapel looks interesting, thanks for the reference, I’ll have a read!
I’m surprised nobody has mentioned Leo. It’s a tree-based text/code editor. The tree is very flexible, and leaf nodes contain code/text/whatnot. You can even reference leafs from more than one place.
It’s not code-specific. That is, you can use it for any text-based programming language (though there will be a step to export linear-text files), or you can use it for documentation, or todo lists, or whatever. It’s very flexible.
I used it a long time back to do some literate programming. It’s evolved quite a bit since then.
Leo is maybe more an outliner than a tree editor – the nuance being that a tree editor should have a notion of schema, while an outliner can be more freeform. They’re definitely two branches of the same family tree, but I feel there are unique challenges when interactively editing structured data that needs to (eventually) conform to a given schema.
That looks quite neat; reminds me of a similar terminal program called hnb, although hnb seems more minimal and more targeted towards personal information management. Leo seems much more poweful and worth exploring, thanks for bringing it to my attention!
An interesting example that seems to be missing is the ABC Structure Editor.
ABC is considered the spiritual ancestor to Python; Guido van Rossum worked on ABC before he created Python.
I find scratch.mit.edu to be somewhat good at this. There is also a paper somewhere in the vissoft2014 publications that shows using types to prevent blocks that don’t fit in a place from appearing there.
One thing to note is that often – I am trying to do an edit that is internediately not an AST, till I get to one. This is often the case when group editing many lines with a regex or such. One clearer variant of it is – intermittently doesn’t typecheck, then will. We need to be careful about which states we let the UI let the user put themselves in and which they don’t; or make it clear how they can express how much they want at each moment.
Piggy-backing on your comment, as its second part conveys more or less what I wanted to reply. I think it’s not my own intuition, that I’ve read it somewhere (probably HN): that there may actually be a reason why people seem to stick to text editors in the end. And it may be that the AST editors are too strict for expressing programmer’s thoughts, which may start on the more vague and imprecise side. For example when writing something, I may want to be able to start by “drafting” the code, to put down a sketch of my general idea, and only then run a compiler and syntax-check or even type-check it, and start molding it to fit into the rigid requirements of code. (Quite often this phase may invalidate some of my initial assumptions and force me to rethink parts of my solution, of the “thesis” expressed in code words.) In other words, I may be ignoring even syntax errors on purpose when writing the initial draft.
On the other hand, this phase could arguably be done in comment blocks / inline comments.
Interestingly to me, this idea seems to have some consequence for me when trying to use the luna-lang.org GUI. On one hand, they seem to be somewhat aware of this dynamic, as the GUI allows adding invalid nodes, and composing them together in correct way only later. On the other hand, I seem to recall that when trying to use it, I had this weird feeling that I still wanted to first sketch the overall structure of a function on a sheet of paper, before trying to “draw” it in the Luna GUI. Though this may as well have been purely because of the slowness of the GUI as of its current iteration. (Keeping my fingers crossed for the new GUI they want to release in the new year — hopefully they manage to push it out the door earlier than later.)
Also, on a more or less related note to the OP, the recent announcement of the “Tree Sitter” approach to parsing may or may not be somewhat relevant here (HN, lobste.rs).
As you point it out, the sweet spot would be for a tree editor to allow unconstrained editing (ie. no enforcing of schema/grammar) as well as constrained editing. Taking the analogy of XML, you could very well edit a tree with multiple namespaces weaved into the document, one validated by a schema, and the other one only loosely validated.
Speaking of GUIs, I think that part of the reason tree editors are still pretty much a research topic is the lack of supporting infrastructure. Programming is very text-centric and so are the tools (starting with UNIX, the shell, etc). Most GUI toolkits lack the flexibility required by interactive, user-friendly tree-editing – it’s only in the past few years that the DOM, CSS and the Web APIs have evolved to a point where this is feasible with good interactivity and reasonable performance (although, as you point out, performance is still an issue for some).
And when you think about it, it make sense: the Web is built on the notion of trees combining pieces of heterogeneous data together. With XML you have XSLT (transformations), XPath (queries) and XML-Schema (grammar). With CSS you can style the presentation and control the interaction with JavaScript.
For me, the takeaway of Tree Sitter is that we’re starting to realize that incremental transformation is the way to go for programming languages tools (parsers, compilers, etc). When you’re editing a live AST, you have the ability to stream tree deltas (patches) directly instead of re-processing the whole thing. All you need is a compiler/interpreter that is able to translate these changes into changes to the output (binary/UI).
With so many programming languages out there and LLVM becoming the de-facto substrate, I feel we’re on the right track to start rethinking syntax and free it from the shackles of a given textual representation.
I’ve been playing around with similar ideas: https://nick.zoic.org/art/programming-beyond-text-files/
That’s a good start, but I feel one of the main issue is navigation (with mouse and keyboard), and the other is dealing with eventual consistency with the schema – it would be interesting to see how you would address these in WASTE. If you’re interested in dataflow and visual languages, I’d recommend reading the corresponding chapter in Wouter van Oortmerssen’s Aardapel thesis. It features some interesting insights into the advantages and drawbacks of visual and dataflow languages.
Thanks! It’s something I’m messing with on and off as time allows, so it’s evolving very slowly.
The general idea was to force the document to always be consistent with the schema by having kind of a typeahead buffer (keystrokes don’t commit to the tree until they are parseable) and/or inserting extra leaves (you type ‘and’ on the end of an expression, it adds a temporary ‘true’ on the end to make that into a proper tree, and lets you overtype that with whatever you were about to say.
I’d like it to work with existing languages eg: Python/Javascript/Haskell because I’m more interested in the user interface aspect than the language design on this particular idea, but obviously that brings its challenges.
Aardapel looks interesting, thanks for the reference, I’ll have a read!
Is anybody in here actively using tree editors?