1. 3

    Last time I checked, Adapton was making heavy use of caching, and it wasn’t clear what the memory overhead would be compared to a full recomputation approach. The shift that needs to happen, I think, is to reify the notion of change in the API or language, so that functions emit changes to data structures and the compiler then combines the changes together, collapsing intermediate steps that are just transient.

    1. 1

      I think that software architecture is an increasingly important field. The approach from the 90s showed its limits (ie. applying the metaphor of building construction to software), but given that creating programs has shifted from a handful of relatively independent, experienced hackers to an army of bootcamp-trained developers, the role of architecture is going to be more and more prominent. The key challenge is the same: how do we approach the design of a computer program, taking into account not only what it should do, but how it’s going to be made.

      1. 4

        The article is very clear and accessible. Now I’d be curious to hear people’s practical experiences in leveraging the properties of algebraic data structures in their programs. It seems similar to UML: a great idea and a potentially impactful tool, which is very rarely used in practice.

        1. 1

          Yeah, I’m really interested in that, too. It seems most interest is either formal verification or people that program in Haskell. I’m wondering how much if any practical value it has outside those groups. That is, outside it just being interesting and mind-expanding.

        1. 2

          Is there some docs on writing the grammar? I like where the project is going (I remember your last post here).

          This may be a silly question, but how do you represent a list of nodes (I see named maps in the examples) and attributes?

          1. 1

            We don’t have good human made docs yet on Grammar authoring. I know some volunteers were working on that, but I don’t think they’ve published anything yet and not sure the timeframe.

            Here’s the best we have at the moment: http://treenotation.org/sandbox/build/#standard%20grammar

            The Grammar language is itself written in the Grammar language. So any keyword you see in “foo.grammar” will be defined in “grammar.grammar”.

            I’m also hoping we will add auto-generated docs by the end of the week (https://github.com/treenotation/jtree/issues/26).

            how do you represent a list of nodes (I see named maps in the examples) and attributes?

            The base node actually has children as a list of nodes, and not a named map. Thank for asking that (a few folks have had questions about that and so it I just created an issue to improve it). Here’s the basic data structure:

            interface TreeNode {
              parent: &TreeNode
              children: TreeNode[]
              line: string
            }
            

            The Stump Tree Language, which compiles to HTML, implements a list of htmlTag nodes with attributes, may serve as a good reference: http://treenotation.org/sandbox/build/#standard%20stump

          1. 5

            The link is broken now https://github.com/DutchmanDavid/DeluxePaint, but it seems to be available at https://github.com/historicalsource/DeluxePaint. I’m quite surprised to see it’s written in C, I would have thought that most of the apps back then were written in assembly.

            1. 16

              I’ve been doing tabs for indent and spaces for alignment for a similar reason: some people like a tabwidth of 2, some of 4, some of 8. Also, using tabs makes parsing slightly easier, and provides visual guides when you represent tabs with glyphs in your editor. But nothing beats this accessibility argument, it makes the choice much more obvious.

              1. 6

                The problem with tabs is that lots of people use tabs in such a way, that it looks sane only when you’re using their tabwidth, and looks confusing if you’re using a different tabwidth. So not only this defeats the purpose of tabs, but you also need to know their tabwidth in order to even read the source code, let alone modifying it.

                1. 1

                  Can you elaborate on this? It should only be an issue if different lines in the same file use spaces on one line, tabs on another.

                  1. 4

                    Here is an example of what I have in mind:

                    http://anadoxin.org/blog/youre-using-tabs-in-a-wrong-way.html

                2. 2

                  Seems reasonable. I still have one problem, though: in the editors I have used to date (emacs, QtCreator, VisualStudio, XCode), I haven’t found an easy way to actually do that.

                  Emacs has the best UI when you agree with the mode’s settings: hitting [tab] will instantly indent and align the current line of code. If you are using tabs however, emacs will use as much tabs as possible, then fill the remainder with spaces. Which is hell with variable width tabs.

                  The other editors, by default would just add one tab character (or 4-8 spaces) when I hit [tab]. No more easy alignment, that’s a drag.

                  It’s been a while since I used a modern IDE, though, and I never tried to have Emacs do the right thing (I only used it on spaces projects) Can we now easily set up our editors to help us indent with tabs and align with spaces correctly?

                  1. 2

                    I just had a quick look in the QtCreator settings (Tools menu > Options > C++ > Edit… > General), and it looks like if you select these options, you will get tabs for indent and spaces for alignment within the indentation level:

                    • Tab policy: “Tabs Only”
                    • Align continuation lines: “With Spaces”

                    I haven’t tested it thoroughly, but it appears to do the right thing in the example code next to the settings.

                1. 3

                  Unpopular opinion ahead: I don’t think this corporate point of view onto the issue solves the actual problem, it only gets it out of the hair of corporate and makes it someone else’s instead.

                  Let’s consider the assumption that an “asshole” can be “fixed”, i.e. will adjust their behavior after being reprimanded with sufficent frequency. As far as I can tell, you haven’t changed their nature, just the symptom. There must be an underlying reason why the person is like this: Be it stress, disliking their job (but feeling trapped into it with no viable alternatives) or possibly inherent malice. If and only if the cause was stress induced by other people at the workplace, this alleviates the cause as well as the symptoms. Otherwise, you’ve merely shifted the problem on someone else, such as their family or random strangers. Possibly, lacking an outlet for their “assholery”, this might even intensify the outlash at others outside a workplace. I’m not sure what would actually fix this; possibly some trips to the psychiatrist and subsequent therapdy and medication?

                  Let’s consider the assumption that an “asshole” cannot be “fixed”. In that case, it is a fixed personality trait, likely rooting in inherent malice. This article makes the point that assholes should not be permitted to work in a workplace. At the same time, the assumption is that they’re irredeemable. Thus, they cannot work in anything but one-person self-employment scenarios, which is an economically unrealistic scenario for many people. Therefore, these people cannot be allowed to work at all according to this line to thinking. What does a society do with people that cannot work? Banishing them out of the country or outright killing them is apparently out of the question for the morals of society right now, so they’d have to join welfare. In a capitalist state, the question turns into whether, on a macroeconomic scale, putting “assholes” on welfare comes cheaper than keeping them in the workforce.

                  In either case, you have an unsolved problem.

                  1. 8

                    “Therefore, these people cannot be allowed to work at all according to this line to thinking. “

                    A tiny percentage of environments that read and apply writing like this won’t hire them. Most environments hire assholes. So, these people will be allowed to work in most environments. Might even thrive in them. They’ll simply go to the asshole-tolerating companies.

                    1. 3

                      you’ve merely shifted the problem on someone else, such as their family or random strangers.

                      They are already assholes to other people in their lives.

                      1. 2

                        It is interesting to consider what you propose from a “corporate social responsibility” perspective: a company might then help society in general by helping resolve “assholistic” behaviour, and offering an environment where the person can improve and eventually resolve or reduce the problematic behaviour. This leads to interesting moral and ethical questions, and ultimately to the role of a company and its indirect social impact.

                      1. 4

                        This looks interesting. The slides are pretty information dense. Is there a talk/video to go along with them ?

                        1. 6

                          Not for that talk, but I think this Ken Scambler talk from LambdaJam 2015 is much better, and I think this slide deck is the one Ken uses, or is very close to it.

                          Whipping out the “monoid in the category of endofunctors” line doesn’t help unless you have explained what those words actually mean - I programmed using monads for a number of years without understanding that catchphrase. I’d also say that OP’s slides probably need at least a lot of talking around to be useful. The slides talk about monad flattening List(Maybe(T)) to List(T), but monadic join is M x M -> M, so that seems off?

                          1. 2

                            Thanks for the links! What I liked about the “Category Theory in 10 minutes” is that it helped me get an idea of what category theory is about and introduce some of the lingo that otherwise takes a long time to acquire (it’s only 22 pages long). As a point of comparison, The Category Theory For Programmers book is 515 pages long, the Seven Sketches in Compositionality book is 243 and the slide deck you shared is 209 – I’d like to see more short or medium-form introductions to category theory, so that it’s easier to dive into it.

                            1. 1

                              I caution you against taking the page count of any slide deck too seriously: often it’s because of little highlights and not-quite-animations. Ken’s talk is only 25 minutes long.

                        1. 2

                          I submitted this not really because I agree with it but was curious if any lobsters out there have strong feelings. I have been using other languages for a while and not stayed up with recent developments. I generally still like ruby and use it fairly regularly and actually kind of like the constancy of it. I do, however, find myself reaching for other tools more often lately.

                          However, the claims here that ruby core is working against the general sentiments of the community at large, while examples were provided, seemed slightly exorbitant. So I was curious if others here felt this is the case or not.

                          1. 10

                            Ruby’s dead. How’s that for a strong feeling?

                            I wish I could be more articulate, but it seems an inexorable conclusion of two forces. One, javascript is the language of the browser. Two, people have been making javascript as easy as possible for decades now.

                            Notice there are a lot of kids on http://repl.it/talk … I wonder how many of them are genuinely trying to learn ruby rather than JS?

                            Look, I’m no JS zealot. I’m more of a “skate where the puck is going to be” kinda guy. And the ruby players on the rink have been skating toward other places.

                            Rails was incredible. I remember how magical that first demo was. You could do in ten minutes what took days in PHP.

                            Nowadays, doing that is sometimes as easy as running now.

                            Oh boy, there I go again expressing super strong confrontational opinions… For what it’s worth, if I’m wrong, you’ll get the satisfaction of rolling your eyes and watching Ruby eat the world over the next decade. But it just doesn’t feel like that’s the world we’ll end up in. Pull up any visualization of “let’s measure the popularity of programming ecosystems” and it’ll look a lot like JS and Python won. https://www.youtube.com/watch?v=wS00HiToIuc

                            1. 10

                              As a ~10 year experienced engineer with the bulk of that time in Rails currently looking elsewhere, I will say: the job market would seem to indicate that Ruby is very much alive and well. I’m curious about the JS comment, as I feel like Ruby has never been the language of the browser.

                              That said, the mid-to-upper tier web companies these days seem to be doing server side development in Go more often than Rails. I’m looking to change only because I want to learn something new. Ruby/Rails if anything has slowed down its formerly frenetic rate of change, performance has increased to a respectable degree and the ecosystem is filled with robust libraries, a good testing culture… Rails is by no means a terrible stack to work in. YMMV of course. Edit: I do agree with the examples in the original article… the pipe operator in particular strikes me as an ugly solution looking in vain for a problem.

                              But I find the “Javascript is taking Ruby’s place” remarks very confusing, as Ruby is a server-side language, and Go seems also to have stolen the server-side market share from Nodejs.

                              1. 6

                                as I feel like Ruby has never been the language of the browser.

                                Whoops. My point was, JS is the language of the browser. If you want to do much of anything with “webpage + browser,” you need to know JS.

                                That means if you know JS, you can largely get by without knowing anything else. Or rather, you can learn whatever else you need to learn as you go.

                                job market

                                We’re so lucky that the job market gives us so many options. I totally agree; I didn’t mean to imply that if you’re a ruby dev, you should worry about your career prospects. So much of the world was built on Rails that you’ll probably be able to find work for a long time.

                                All I meant was, younger people don’t seem to be interested in learning Ruby. When those younger people become older people, and the older people become less and less people, the world changes.

                                If that sounds grim, just be glad you’re not a Lisp guy like me. It’s almost painful to watch everything not use the power of compile-time macros. But at least I get to use it myself.

                                Go seems also to have stolen the server-side market share from Nodejs.

                                You’re right that Go has had some surprising momentum here. Much of repl.it is apparently built on Go. But the advantage of JS is the ten hundred million libraries that exist to solve all problems ever thought of. (More than a little bit of hyperbole here, but it’s almost not far from the truth.) If you need to do X and you happen to be using JS, you don’t have to read any docs. You can just type “do X in Javascript” into google, and google turns up an npm package for X, for all basic X. Other languages will always be second-place regarding convenience, for this reason.

                                Super Serious Projects will tend to be written by people who want absolute type safety and clearly-defined module boundaries and never to see an error. Hell, golang doesn’t even have a REPL. But anyone who’s missed a REPL will tell you that it’s a serious disadvantage not to have a REPL.

                                1. 9

                                  the advantage of JS is the ten hundred million libraries that exist to solve all problems ever thought of

                                  As someone who has done quite a bit of both Go and JS, this is emphatically not a point where JS wins.

                                  There’s a bigger number of packages, but bitter experience has not been kind to my trust in ‘this JS package has lots of downloads and github stars and a nice website so it probably works OK’.

                                  1. 1

                                    “worse is better, lol.” https://www.jwz.org/doc/worse-is-better.html

                                    Most people see a package and expect a solution. But each package solves N% of whatever problem you’re facing. (N is usually somewhere between -10 and 97.)

                                    As much as I love to write code, I love getting things done quickly without introducing major problems. npm i foo tends to work pretty well for that.

                                    Hey, cool trick. npm install sweetiekit-dom and then do require('sweetiekit-dom')(). Now window, document, etc all exist, just like node is a chrome repl. You can even do window.location.href = 'https://google.com' and it’ll load it! console.log(document);. Unrelated to the convo, but I can’t get over how neat it is.

                                  2. 3

                                    So I decided to test “do soundex [1] in Javascript” just to see if it was true. Yup, second entry on the results page. I checked it out, and found an error—“Ashcroft” encodes to A261, not A226. And given the page it’s on is a gist, there’s no way to report an error.

                                    [1] Why Soundex? Well, I used it years ago in an online Bible to correct Bible book names (such that http://bible.conman.org/kj/estor will redirect properly to http://bible.conman.org/kj/Esther).

                                    1. 1

                                      bible.conman.org

                                      Example URL, or satire?

                                      1. 1

                                        Neither. When I originally registered for a domain back in the late 90s, I wanted conner.com but that one was taken. So were conner.net and conner.org. My backup choices, spc.com, spc.net and spc.org were also taken. I had a few friends that called me Conman (a play on my last name of Conner) so that’s what I went with.

                                        In the 21 years I’ve had the domain, you are the first one to question it. No one else has (Weird! I know! [1]). The link is real, try it.

                                        [1] It’s also weird how few people connect my name, Sean Conner, to the actor Sean Connery (one letter from stardom!) At least my name isn’t Michael Bolton.

                                        1. 1

                                          That’s fine. I just reacted to the domain, and in these contentious times it’s not too hard to imagine a person setting up a Bible site with pointers to the “bad stuff” (depending on your view of what’s bad).

                                          FWIW I”ve used https://www.biblegateway.com/ a few times (mostly because I’d be interested in how the text is presented in different Swedish editions) but that’s an altogether bigger operation.

                                  3. 6

                                    Agreed. I would hypothesize that the Ruby community is largely being cannibalized by: Go, node.js, Elixir/Phoenix, Rust, Python (for special purpose work like tensorflow) – probably in that order (or maybe swap Rust and Elixir? unsure).

                                    1. 10

                                      It’s not only due to new tech stacks emerging. Cultural and commercial factors play a massive role.

                                      For instance: Ruby got very popular in the consulting space (it’s a good way to look good by quickly delivering some value, and it tends to generate a need for lots more consulting hours a year or so down the track).

                                      Now that the ruby community has more-or-less settled on a few standard approaches, it’s no longer as profitable for consulting companies.

                                      1. 4

                                        I don’t agree fully with that reading, Rails was always also very popular in the old-school agency space, as Rails is extremely quick in getting set up. It’s insistence on having a standard stack might lead to problems in the long run, but still makes it the best framework for quickly getting out of the door in a clean fashion.

                                        It still remains very popular there.

                                        Also, Rails is often used for internal management applications, I have tons of clients that “don’t do Ruby” until slowly, you figure out there’s tons of small applications running on their servers essentially providing some buttons and graphs.

                                        The number of companies that “don’t do Ruby” officially, but actually do internally is huge, especially in enterprise.

                                        1. 1

                                          That’s a great perspective, thanks for brining it up!

                                        2. 5

                                          Speaking from the perspective of someone who is both in the Rust project and on the board of one larger Ruby non-profit, I do not agree with the point that Rust cannibalises Ruby. Indeed, we’re still growing, even if the curve flattens.

                                          1. 1

                                            I only have a limited set of data points for folks I know of that have moved (or are moving) from ruby to rust for a couple of projects (blockchain space). Sounds like you have more empirical evidence here for sure.

                                            1. 3

                                              Rust is pretty popular for implementing blockchains, and Ruby isn’t, because you can’t write a competitive PoW function on top of the mainline Ruby implementation. Most Ruby projects don’t need that kind of performance, so your story probably isn’t very typical.

                                              1. 1

                                                Experienced developers usually extend their toolchain at some point, coming with a shift of interest. There’s an effect where you have more insight into see experienced people picking up new stuff, but tend to ignore newcomers coming up.

                                                I am of a certain generation in the Ruby community, which leads to the good effect that a) I meet more and more people that don’t know me, despite having a high profile, b) I tend to only see my circles and have a hard time keeping track of newcomers.

                                              2. 1

                                                I agree, and I think they complement each other more than compete right now. Ruby is great at building architecturally slick webapps, which Rust is lousy at. Rust is great for building high-performance low-level stuff, which Ruby is lousy at. It seems like a good pattern, supported by several gems/crates, to build a webapp in Ruby/Rails, and refactor any parts that need top performance out into a Gem written in Rust.

                                              3. 2

                                                I very much doubt Ruby devs are moving to a language as low-level as Rust.

                                                Elixir I could very much believe.

                                                1. 8

                                                  I very much doubt Ruby devs are moving to a language as low-level as Rust.

                                                  Roughly 1/3rd of the Rust programming language community come from dynamic languages, mostly Ruby and Python.

                                                  1. 1

                                                    How do they deal with lifetimes? Whenever I use Rust, I tap out at lifetimes because it just gets too confusing for me.

                                                    1. 4

                                                      The zen of Rust is using Ownership in most spaces. Lifetime problems usually arrive when you are trying convoluted structures that are better handled through cloning and copying anyways. Use clone() liberally until you are very sure of what you want to do, then refactor to lifetimes.

                                                      I wrote a glimpse into this here last year: https://asquera.de/blog/2018-01-29/rust-lifetimes-for-the-uninitialised/

                                                      Also, Edition 2018 made lifetimes a lot easier.

                                                      1. 2

                                                        Thanks for the link! :)

                                                        1. 1

                                                          You’re welcome!

                                                    2. 1

                                                      i’m pretty excited about the mruby/rust integration, especially if i can eventually ship a single executable with embedded ruby code.

                                                      1. 1

                                                        Is that being talked about anywhere? I’d love to follow that conversation as well

                                                        1. 1

                                                          i know about mrusty but it seems to not be active; i’m just hoping that people are still working on this (i might even join in if i get some free time)

                                            2. 8

                                              I have never used Ruby in anger, but gosh that Immutable Strings bug getting closed out as “not going to do it, don’t care you all want it, just use a magic comment” would make me think that the Ruby you’ve got is the Ruby you’ll ever get.

                                              I don’t think that languages have to keep being developed (how many Lisp dialects are there that don’t change?), but if you think Ruby has deficiencies now, I wouldn’t expect them to change and that would make me worried too.

                                              1. 7

                                                I am maintaining a ruby codebase that’s >10 years old.

                                                I don’t want ruby to make backwards-incompatible changes! The language is established now; it’s far too late for that.

                                                It sucks that you need a linter to check your files start with a magic comment in order to get sensible behavior, but not nearly as much as not being able to upgrade & benefit from runtime improvements/security patches just because they’ve changed the semantics of strings for the first time in 25 years.

                                                1. 6

                                                  This is an awful sentiment. How would you like being told that for a project you maintain, you can no longer make any big changes, ever? Because some user from 20 years ago doesn’t want to update their scripts, but wants bleeding edge Ruby.

                                                  The world doesn’t always work that way, and hopefully Ruby doesn’t listen to people like that.

                                                  1. 8

                                                    I actually think it’s a pretty reasonable statement. One of my favorite things about Java is that it’s almost 100% backwards compatible. We just dusted of a 20 year old (!) game and it pretty much worked on that latest JDK. That’s awesome.

                                                    1. 1

                                                      So is C, and C++ and other natively compiled languages. The advantages of a standardized lower layer!.

                                                    2. 5

                                                      If you want to maintain a project where breaking things to make other things better, find one where the things you break don’t affect people. There’s no shortage of them and it’s even easy to start your own!

                                                      If you want to be the trusted steward of a large community, you have to consider how your choices affect the people who have placed their trust in you. There’s nothing wrong with not wanting that! It’s a lot of work. I don’t want it either. Thankfully, some people do, and that’s why communities have some people at the center and others at the periphery. The ones at the center are the ones doing the hard work of making it possible.

                                                      1. 2

                                                        Hopefully they do. It’s great to have new language features and to advance the state of the art, but it’s also great to be able to run my code from a few years ago without having to rewrite it.

                                                        There are ways to have both, of course, which involve making compromises. For example, in the area of scientific computing I’m currently working in, there are a lot of Python 2 hold-outs who don’t want to migrate to Python 3, even though the changes are few* and Python 2 support is due to end. But many Python programmers are happy with Python 3 and have ditched 2 altogether already.

                                                        *few, but important in context: changing how division works is a big deal for numerical simulations.

                                                        1. 1

                                                          maintainers who don’t want to be told that should not maintain languages

                                                          1. 1

                                                            This kind of thinking is how you get things like Python 3 being out for over a decade while some people still do everything in 2. If you intend for your language to be widely used, you have to come to terms with the fact that even minor changes that are highly beneficial will be massively painful, and might even destroy the language entirely, if they break old code.

                                                          2. 3

                                                            Python 3 actually introduced breaking changes, which in hindsight were all really good. I had to convert dozens of projects over a couple of years, it was not that bad once I understood how things worked in Python 3. The biggest change was the fact that strings are now Unicode strings instead of ascii, and it was very confusing at first.

                                                            1. 1

                                                              IMO python 3 is a great example of why I’m glad I don’t maintain any python codebases, despite loving the language.

                                                              In a maintainer-friendly world, developers would still have to write a bunch of from __future__ import X at the top of every file today, which sucks differently but IMO not nearly as much. If you were somewhat forward about it, files that don’t have those lines could emit deprecation warnings when loaded warning that those defaults will be enabled in a few more years time.

                                                        2. 1

                                                          I’m sure that a lot of decisions in Ruby in the past were questionable, I just didn’t know about them before I started learning Ruby. However, now that I keep an eye out for programming languages in general, I feel like it’s made me a bit of a snob. I tend to agree with the author of the blog post that it puts a bad taste in my mouth for the language to be changing like it is (both the language itself as well as the process in which those changes are happening) but I’m not sure these things would have bothered me if I were coming to it as a new programmer like I did with Ruby 1.9.

                                                          I made a comment somewhere that lamented that Ruby was adding Enumerable#filter because it was ambiguous whether it was equivalent to #select or #reject. The response I got was that it was a good change because that’s the way that every other language did it. Ruby’s just kind of weird sometimes, and I think I have accepted the a lot of the legacy weirdnesses. So in that respect, what’s one more feature I won’t use?

                                                          In the end, I don’t have much stake in the game - if Ruby’s new path really starts to bothers me, there are plenty of other languages to pick up. But until then, it will be the first language I turn to for quickly translating thought into code, weird language design cruft aside.

                                                          1. 5

                                                            Ruby 1.9, in hindsight, was extremely well managed. It was an opt-in to breakage for getting fundamental problems out. They handled that switch in a very good way, making Ruby 1.9 the clearly better version while releasing 1.8.7, which closed the gap in between both versions, making it feasible to write codebases that run on both with relative ease. Sure, there were issues and not every aspect was perfect, but comparing e.g. the Python 2.7/3.0 story, I’m sad that the Python community hasn’t been watching and learning from that.

                                                            1. 1

                                                              Agreed, and I find Python’s rise in popularity comes in spite of the poor developer experience - compatibility and dependency management - so I wish Ruby had made more headway in non-Rails contexts.

                                                            2. 2

                                                              I made a comment somewhere that lamented that Ruby was adding Enumerable#filter because it was ambiguous whether it was equivalent to #select or #reject.

                                                              Agreed. select and reject is a naming choice I have decided to steal, I wished filter just stopped existing (or returned (selectedElements, rejectedElements).

                                                          1. 2

                                                            This is very interesting, especially the fact that you have an expressive and compact notation, a grammar format and tooling around it. I don’t know if you heard of Loyc Trees, but it has a somewhat similar good (Loyc is more focused on programming languages AST as opposed to a generic tree format like yours).

                                                            I’m not sure if you wanted to say XML instead of HTML, but something I really like about XML is the fact that you can weave in elements from different semantic domains using namespace (an obvious example in HTML is the embedding of SVG). XML also comes with validation (XMLSchema, Relax-NG, etc) selection (XPath) and transformation (XSLT), which makes is possible to extract and transform (relatively) easily. Your grammar language seems to do tree to arbitrary structure (but primarily text), so I was wondering if you had special support for tree-to-tree transforms. For instance, one could use XSLT to converting DocBook to XSL:FO and then render it in PDF.

                                                            I’ve done a fair amount of XML and XSLT, as you can guess, and have to say the verbosity and overall clunkyness of both the syntax and API so I’m interested in anything that can replace that toolset.

                                                            1. 2

                                                              I’ve done a fair amount of XML and XSLT, as you can guess, and have to say the verbosity and overall clunkyness of both the syntax and API so I’m interested in anything that can replace that toolset.

                                                              Same here.

                                                              1. 1

                                                                As far as I can tell from my notes I have never seen Loyc. Just browsed through it a bit. Interesting stuff.

                                                                In this particular post I was specifically talking about Stump as a preview of a successor to HTML. But more generally yes, Tree Notation can replace XML in many places.

                                                                but something I really like about XML is the fact that you can weave in elements from different semantic domains using namespace

                                                                This is one of my top 3 priorities right now with Tree Notation: grammar composition. I’m actually hoping to have it launched in the new few days.

                                                                so I was wondering if you had special support for tree-to-tree transforms

                                                                I do not have a DSL yet specifically for doing tree transformations. It’s something I’ve played with in the past but never came up with anything I was crazy about. Currently I just have methods implemented in the TypeScript library for doing most of the tree op stuff. What might be a solid starting point is an ontology of all the general types of operations one would want to do on Trees.

                                                                selection (XPath)

                                                                Similarly I’ve played with languages for querying, but haven’t come up with anything special. I hope someone might come up with something neat here. Right now I just have an API on the library for doing things like select() and where().

                                                                validation (XMLSchema)…

                                                                My Grammar Language (http://treenotation.org/sandbox/build/#grammar) is my version of this.

                                                                I’ve done a fair amount of XML and XSLT, as you can guess, and have to say the verbosity and overall clunkyness of both the syntax and API so I’m interested in anything that can replace that toolset.

                                                                Awesome! I would also greatly benefit from that sort of thing. Do you think it’s best implemented as its own DSL or in a libraries in a traditional general purpose programming language?

                                                              1. 8

                                                                Maybe I’m missing something, but I don’t see how this is the first draft of an HTML successor? Isn’t this just a meta language to abstract over HTML in its current form?

                                                                1. 2

                                                                  Today this compiles to HTML.

                                                                  But in the future browser engines could just parse this directly. I also have one for CSS called Hakon, http://treenotation.org/sandbox/build/#hakon, and there are a few in progress for Javascript. So in the future you could have 1 syntax for all the languages. Your HTML IDs would leverage the same typing information used by your CSS and your Javascript–no more bugs due do having to duplicate that kind of information.

                                                                  So I see this in the long term as the beginning of the successor to the HTML/CSS/JS stack.

                                                                  1. 12

                                                                    That looks like a DTD, except with a Pythonesque syntax instead of SGML.

                                                                    I don’t see where there’s value in it for HTML. It may be a neat macro language, but it seems quite presumptuous to call it a successor. HTML survived browser wars, pre-WHATWG useless specs, XHTML and XHTML2. It’s used for interoperability and compatibility with the massive amount of existing content. Such strong network effects are generally impossible to break with same-but-better things.

                                                                    1. 1

                                                                      Thank you for the comments and all very fair points.

                                                                      “HTML survived browser wars….”

                                                                      HTML, created by Berners-Lee, came out around 1990. XHTML in 2000. So HTML is about 30 years old.

                                                                      Binary Notation, as formulated by Leibniz, came out in 1689. So over 300 years old.

                                                                      Tree Notation is more like “Binary Notation” than it is a programming language. Just as binary is the simplest way to encode information, Tree Notation is the simplest way to encode abstraction. You cannot improve upon it, and in trying to improve upon HTML/XML/Lisp/any other language people will keep reducing until they reach Tree Notation.

                                                                      HTML survived for 30 years but I don’t think in 30 years we’ll be using it. We’ll have switched to Tree Languages. (Or perhaps, in 30 years will have switched to Tree Languages and then to something else even better that I can’t foresee).

                                                                      “Such strong network effects”

                                                                      Tree Notation will also have very strong network effects over time. A few years ago it was just an idea, but now in an hour you can create a whole new Tree language in Tree Notation and get a parser, compiler, VM, syntax highlighting, autocomplete, program synthesis, type checking, visualization, and a whole lot more.

                                                                      “it seems quite presumptuous”

                                                                      When you discover what is comparable to a new fundamental law of physics, and confirm your discovery over many years with immense amounts of data and experiments, you have 3 options: 1) don’t tell anyone, 2) tell people but lie and say it will not cause a massive paradigm shift 3) tell people the truth and be accused of madness/arrogance/presumptuousness. I’ve decided option 3 is the lesser of 3 evils. It’s a long and lonely road but a few people have been immensely helpful and we are making great progress!

                                                                      1. 11

                                                                        You cannot improve upon it,

                                                                        Reminds me of the kind of language used in all those advertisements for quack medicinal products in centuries past.

                                                                        HTML survived for 30 years but I don’t think in 30 years we’ll be using it. We’ll have switched to Tree Languages.

                                                                        This is so incredibly off-base it verges on delusion.

                                                                        in an hour you can create a whole new Tree language in Tree Notation and get a parser, compiler, VM, syntax highlighting, autocomplete, program synthesis, type checking, visualization, and a whole lot more.

                                                                        I don’t think anyone would want to use them, and it’s not possible to generalise the creation of usefully distinguishable VMs.

                                                                        There are no actual evils in your three options, but the third will certainly lead to people ignoring you. If you consider the opposite of evil to be people enjoying and appreciating your creation, then it’s definitely not the lesser of the three; try option 2.

                                                                        1. 2

                                                                          Reminds me of the kind of language used in all those advertisements for quack medicinal products in centuries past.

                                                                          I think this is a very healthy attitude to have. Probably 99/100 times someone says something like that they are full of it, so it is safe to ignore them. I’m not asking you to believe. I’m asking people to try your best to prove me wrong.

                                                                          Can you improve upon binary notation? Can you come up with a simpler way to represent all information than just using 2 markings? If you can, you may win a Nobel or Turing Prize. If not, then it shouldn’t be too controversial to say that binary notation is the best notation for storing information.

                                                                          Likewise, can you improve upon tree notation? Can you come up with a simpler way to represent all abstractions than just using horizontal and vertical spacing? I can’t, have been trying for 7 years, and at this point have code that turns more than 10,000 different languages into Tree Notation, covering hundreds of languages concepts. If you can figure out something better, please, please, share it with me so I can move on to other things and you may win a Nobel or Turing Prize.

                                                                          This is so incredibly off-base it verges on delusion.

                                                                          You are right! E pur si muove!

                                                                          I don’t think anyone would want to use them

                                                                          This is a great criticism, and my only answer is we are making progress! The Tree Language powering Ohayo is getting quite good, and could be the first killer language, in a few weeks to months. I did underestimate (by 10x+) the amount of infrastructure and data we needed to build before we had killer-app Tree Languages. It is not as simple as coming up with a set of words that compiles to the language of interest. That does not get you a great language. Even though the base of Tree Notation is so simple, you still need to think through dozens to hundreds of design decisions to make a great language. Luckily, we’re automating a ton of that through a huge data gathering and machine learning effort. More to show on that front in 6 - 9 months.

                                                                          it’s not possible to generalise the creation of usefully distinguishable VMs.

                                                                          This is a good insight. However, we can indeed make the building of “usefully distinguishable VMs” 10x easier through both the rules of the language itself coupled with a huge dataset of languages, machines, and patterns.

                                                                          There are no actual evils in your three options,

                                                                          Yes, there are.

                                                                          1. If I tell no one, people die, diseases remain harder to cure, and a whole host of other problems remain more difficult because we are wasting incredible amounts of time and energy using suboptimal tools.
                                                                          2. If I tell people but say “this only applies to some small domain”, not only am I going against my code of being honest, but we miss out on the huge network affects that come from people utilizing the universality of Tree Notation, leading again to problem #1 above.
                                                                          3. Personally it’s not fun to invest so much of my time and resources into this, but there’s nothing I can do more that would have a more positive impact on the world than this. Until other people “get it”, and the community grows, I feel a responsibility to keep pushing the ball forward.

                                                                          the third will certainly lead to people ignoring you.

                                                                          People ignore Tree Notation (and me) no matter what. I’m not worried about the majority of people. They’ve got their own lives to worry about. My strategy is to reach people who understand “exponential thinking”. If there’s a 1% chance I’m right, and let’s say software productivity goes up 30% in 10 years. We are talking about an annual value in the trillions. Even at 99% odds of me being wrong this is a many-billion-dollars idea in expected value for the world.

                                                                          1. 5

                                                                            One negative side-effect you may be missing is that people might ignore you and tree notation who would otherwise be interested. E.g. I’m curious about the word “bern” in your example – does that have meaning? But I’m guarded against engaging and asking because I’m worried I’ll have a crazy person following me around and not going away.

                                                                            1. 3

                                                                              I’m curious about the word “bern” in your example – does that have meaning?

                                                                              It’s a placeholder for that keyword that I added a long time ago and haven’t revisited. At the time I couldn’t think of a word for “content” that wasn’t so generic and so went with a unique word that will be easy to find/replace later. Looking at it now my first though was a reference to Bernie Sanders, but then looking back I realized it was a nod to Tim Berners-Lee, which makes more sense. Anyway, I added a todo to the Grammar file for Stump with a clarification there.

                                                                              But I’m guarded against engaging and asking because I’m worried I’ll have a crazy person following me around and not going away.

                                                                              Haha this made me laugh out loud. (And then cry a little bit inside.)

                                                                              It is true though. I strive to be a good scientist. Marketing and community organizing are not my thing. I often jump on the chance to have a conversation about an exponential phenomenon which online can come across as crazy (in person I can quickly assess if someone wants to talk exponential or the more calm here and now). The best decision I made at my last startup was removing myself from the CEO role and recruiting a guy to take over the business and marketing so I could just focus on the code and science. With the Tree Notation Lab I started, I hope to do the same–find a good manager and a good community organizer to lead the “people” efforts so I can focus on the science.

                                                                        2. 6

                                                                          You cannot improve upon it, and in trying to improve upon HTML/XML/Lisp/any other language people will keep reducing until they reach Tree Notation.

                                                                          There have been many proposals over the years for “improving” upon s-expression notation for Scheme and Lisp, including indentation-based “tree notation”. This has never caught on. In the Ruby/Rails scene there has been HAML for a long time, which is basically this notation. That never really became massively embraced by the Rails community either. Sass did catch on massively, even outside the Ruby community, but only after it grew a more traditional CSS-like syntax.

                                                                          Given this history, I have no reason to believe yet another indentation-based notation for HTML is going to change the world.

                                                                          Besides, there are downsides to indentation-based notation for communication. I experience this several times a week when I try to copy/paste some Python code and the indentation gets lost in translation. At least with s-expressions the layout may be messy, but it’ll still work when you copy and paste stuff.

                                                                          1. 2

                                                                            There have been many proposals over the years for “improving” upon s-expression notation for Scheme and Lisp, including indentation-based “tree notation”

                                                                            I currently track 38 of these, which is a shockingly small number of languages (~99% of languages are not indentation-based). The one closest to Tree Notation is I-Expressions, created in 2003 by Egil Möller. It has not gained traction, yet.

                                                                            The difference between Tree Notation and these, like HAML/SASS/etc, are network effects. Currently if I write a tool for editing HTML, it only helps HTML authors. But if I write a tool for editing a Tree Language that targets HTML, it also helps people using Tree Notation to target assembly, C++ , TypeScript, Java, et cetera.

                                                                            Most of my time goes into work on some new very high level Tree languages. All of the syntax highlighting, autocomplete, error suggestions, etc, that are now present in this Stump HTML language, weren’t built for that at all. That’s why these indentation languages will be different than the 38 that have come before.

                                                                            1. 3

                                                                              I’d be very interested in the references you mention, do you have a document somewhere that lists them?

                                                                              1. 2

                                                                                I don’t have it online yet (I’m slowly getting around to open sourcing my programming language database), but here they are as CSV:

                                                                                id,wikipedia,appeared,website,reference
                                                                                abc,ABC_(programming_language),1980,,
                                                                                aldor,Aldor,1990,http://www.aldor.org,
                                                                                boo,Boo_(programming_language),2003,https://github.com/boo-lang,
                                                                                buddyscript,BuddyScript,2002,,
                                                                                cobra,Cobra_(programming_language),2006,http://cobra-language.com/,
                                                                                coffeescript,CoffeeScript,2009,http://coffeescript.org,
                                                                                csl,,1966,,https://doi.org/10.1093/comjnl/5.3.194
                                                                                curry,Curry_(programming_language),1990,,
                                                                                elixir,Elixir_(programming_language),2011,https://elixir-lang.org,
                                                                                f-sharp,F_Sharp_(programming_language),2005,http://fsharp.org,https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/index
                                                                                genie,Genie_(programming_language),2008,,
                                                                                haml,Haml,2006,http://haml.info/,
                                                                                haskell,Haskell_(programming_language),1990,https://www.haskell.org,
                                                                                i-expressions,,2003,,https://srfi.schemers.org/srfi-49/srfi-49.html
                                                                                inform,Inform,1993,http://inform7.com/,
                                                                                iswim,ISWIM,1966,,
                                                                                literate-coffeescript,,2013,https://coffeescript.org/#literate,https://news.ycombinator.com/item?id=5277916
                                                                                livescript,LiveScript,2011,,
                                                                                madcap-vi,,1972,,https://semanticscholar.org/paper/edb9d7f3c18ad47f1c4879434a784a3d17dbea6f
                                                                                madcap,,1960,,https://semanticscholar.org/paper/4bef4a1b67016e569ca7ef57bfc356829c2152ba
                                                                                makefile,Makefile,1976,,
                                                                                markdown,Markdown,2004,,
                                                                                miranda,Miranda_(programming_language),1985,,
                                                                                nemerle,Nemerle,2003,http://nemerle.org,
                                                                                net-format,,2006,,https://genome.ucsc.edu/goldenPath/help/net.html
                                                                                nim,Nim_(programming_language),2008,https://nim-lang.org/,
                                                                                occam,Occam_(programming_language),1983,,
                                                                                org,Org-mode,2003,https://orgmode.org/,https://orgmode.org/org.html#Document-structure
                                                                                promal,PROMAL,1986,,
                                                                                purescript,,2013,http://www.purescript.org/,
                                                                                python,Python_(programming_language),1991,https://www.python.org/,https://www.programiz.com/python-programming/keyword-list
                                                                                restructuredtext,ReStructuredText,2002,,http://docutils.sourceforge.net/rst.html
                                                                                sass,Sass_(stylesheet_language),2006,http://sass-lang.com/,
                                                                                scss,,2006,http://sass-lang.com/,https://en.wikipedia.org/wiki/Sass_(stylesheet_language)
                                                                                spin,,2006,,https://learn.parallax.com/educators/teach/spin-programming-multicore-propeller
                                                                                stylus,Stylus_(stylesheet_language),2010,http://stylus-lang.com/,
                                                                                xl-programming-language,XL_(programming_language),2000,,
                                                                                yaml,YAML,2001,http://www.yaml.org/spec/1.2/spec.html#Introduction,
                                                                                
                                                                                1. 3

                                                                                  There’s also wisp, which seems to be a separate project from sweet-expressions.

                                                                                  1. 3

                                                                                    Another nice catch! Thanks. Flipped that bit in my DB. Looking at this now, I’m realizing that my crawler/ML code was not pulling out indentation languages and that bit was only getting set manually over the years.

                                                                                  2. 3

                                                                                    This list is missing sweet-expressions aka SRFI 110 aka t-expressions, which is an evolution on i-expressions (be sure to read the design rationale, it has some criticism of i-expressions you might want to consider).

                                                                                    1. 1

                                                                                      Good catch, thank you. I track Sweet Expressions but mistakenly omitted the hasSemanticIndentation? flag.

                                                                                      I actually think Sweet Expressions were a regression on I-Expressions.

                                                                        3. 2

                                                                          Hakon doesn’t work correctly. Try setting border to value “1px solid red”.

                                                                          I’m probably missing something, but this looks more like a proof of concept than a proof of an idea to me.

                                                                          1. 1

                                                                            I wrote this in Hakon:

                                                                            .someClass
                                                                             border 1px solid red
                                                                            

                                                                            It compiled to this:

                                                                            .someClass {
                                                                              border: 1px solid red;
                                                                            }
                                                                            

                                                                            Is that incorrect?

                                                                      1. 1

                                                                        I use fish and have the path on the LPROMPT and version control/environment settings on RPROMPT, see here it looks like this:

                                                                        ~/W/tlang »                                        ⏲default|1 ⑂ R26+|26 ⋐ tlang
                                                                          |                                                  |       |     |   |    |
                                                                         CPATH                                      CUR BRANCH       | CUR REV |    |
                                                                                                                          BRANCH COUNT LATEST REV   |
                                                                                                                                       APPENV PROFILE
                                                                        

                                                                        The idea is to put contextual information in the RPROMPT that shows the environment I’m working in, which in turn helps prevent mistakes (ie. I’m working off the wrong branch, I don’t see an available update, I’m working off the wrong environment).

                                                                        1. 10

                                                                          looks like C with slightly different syntax, tbh. I mean, sure, it has defer like golang, but it’s also got manual memory management, null pointers, arbitrary casts, all of which make me not want to use it.

                                                                          EDIT: and that’s fine, a lot of stuff about C’s syntax is wonky, so I totally understand wanting to update it. But given the number of lower-level languages being developed now that tackle some of C’s thornier problems, it seems odd to stop at the syntax level, instead of going further and fixing some of C’s semantics issues

                                                                          1. 2

                                                                            No kidding. It’s even got sprintf(), and presumably the buffer overflow hazards that go with it.

                                                                            1. 1

                                                                              Yes, that’s pretty much the impression I had, but if you look at Tangent, which is a game developed in Adept, it does actually feel more like a lower level JavaScript. It has a scripting language feel to it.

                                                                              1. 3

                                                                                yea, looking at that and it seems very, very golang-like. Which, again, fine if you like that. But “Looks like Go and (maybe) still has all of C’s security issues” isn’t something I’m super interested in trying ¯\_(ツ)_/¯

                                                                            1. 1

                                                                              I’ve been using ranger almost daily for a few months, and really like how it complements a regular GUI file manager. It’s definitely worth trying!

                                                                              1. 1

                                                                                I don’t think this is a useful link, it’s just a link to a “sign up for more info!” page. I’d prefer a direct link to the repo, or to the paper.

                                                                                1. 4

                                                                                  The site actually includes links to both, so it is arguably more useful than a link to either.

                                                                                  1. 2

                                                                                    I agree that the form is a bit prominent, but it gives a good entry point to the different resources, of which the Git repo is admittedly the most interesting at the moment – but the repo doesn’t have a nice rotating diagram ;)

                                                                                  1. 7

                                                                                    Score one more for OTP. It’s such an amazing platform. I’m happy it gets more attention now that Jose Valim dressed it up in a cute dress and called it Elixir.

                                                                                    1. 6

                                                                                      Looking back at the challenges telcos had back in the 80s-90s with more software coming into their hardware platforms, Ericsson made gutsy and insightful decision that still pay off today. Understanding that immuatibility was the easiest way to do (distributed) concurrency and applying it pervasively to a new language was a strike of genius.

                                                                                    1. 4

                                                                                      While the title is misleading, the contents is interesting from a PL design perspective. I find it very regrettable that instead of focusing on simplifying JavaScript and leveraging a rich ecosystem of transpires, the language instead evolved to become some kind of dynamic Java or C#. In an alternate reality, JavaScript got extended with type annotations and JIT hints to make it faster, and got all the support to leverage it’s prototype-based object model. It’s such a missed opportunity that JavaScript became more like Java and less like Lisp (or rather, Scheme).

                                                                                      1. 1

                                                                                        I wonder if using a tree structure (parsed AST) instead of text would make the problem simpler for working on source files. Concurrent edits within a leaf would be just as difficult, but maybe the fact that in other cases you have structural and possibly meta information might make it easier to do the right thing.

                                                                                        1. 2

                                                                                          Doing CRDT on tree structures is much harder than on linear arrays.

                                                                                        1. 3

                                                                                          Can someone explain what ppx means? I feel I’m missing some context to read that post.

                                                                                          1. 3

                                                                                            PPX is an OCaml language feature for syntactic extensions. See http://ocamllabs.io/doc/ppx.html.

                                                                                            1. 1

                                                                                              Thanks!

                                                                                            1. 2

                                                                                              This seems a bit exaggerated, TBH. If Pandoc doesn’t cut it, you can always scrape the HTML and use XSLT to produce the target markup. It’s a weekend of work for a CS grad. But it’s a good reminder that any application should have a “data liberation” feature.