I’m skeptical of the grandiose claims this project makes about making programming meaningfully more accessible to the world than it already is; and I dislike the implication that a webapp is the only sort of software that you might want to use a programming language to build.
That being said, I think that some of the ideas presented here, like trying to make calls to external HTTP APIs look identical to ordinary function calls, or making things like deploy environment and feature flags and database schemas and indices first-class language constructs, are interesting if they can be made to work. But this post isn’t going into enough detail about those neat-sounding ideas to evaluate them yet.
I dislike the implication that a webapp is the only sort of software that you might want to use a programming language to build
Gotta start somewhere, and this is our somewhere :) Ultimately, we’d love to support building all sorts of things, but this is the area we see the biggest painpoint that we think we know how to solve.
But this post isn’t going into enough detail about those neat-sounding ideas to evaluate them yet.
For sure. The APIs as ordinary fn calls worked ok in a naive form, but it still needs a lot of work (esp around reliability in a distributed system, but also language support for optional params and union types). The deploy thing works really great and people love it when they’re starting their app, but it needs more support for people with stable applications, esp from feature flags. With feature flags, our first two versions weren’t great, and we believe we need more tools around running both paths simultaneously, and integration with our user module, to really make it good. The DB indexing we haven’t got enough load for it to have reached the top of our priority list, but migrations are and we’re pretty confident in our solution there.
That being said, I think that some of the ideas presented here, like trying to make calls to external HTTP APIs look identical to ordinary function calls . . . are interesting if they can be made to work.
This was the promise of CORBA, and its abject failure taught us that this goal is neither possible nor desirable.
edit: I guess it’s possible to stake out a position of, roughly: we’ve learned a lot since CORBA, and think we can do it right this time. But I’m skeptical, and I’d like to see at least some academic response to e.g. the fallacies of distributed computing in any such claim.
CORBA said “let’s make remote objects and pretend they’re local objects”. We’re saying “there’s all these 3rdparty HTTP APIs, and they’re all vaguely RESTy, can we get a joint abstraction that incorporates retries, rate limiting, auth, network errors, etc”, and add language features that make these really easy to handle.
Most importantly, we don’t have any control of the other end of the APIs, so all we can do is make it nicer than the current (awful and disjoint) experience.
First of all, it needs to be noted that it’s an announcement for a yet unreleased project.
That said, IIUC the author created CircleCI (?), which would lend him at least a bit more initial credibility than in case of some Joe Random Hacker in a similar situation. Some history of successful delivery reduces the chances of this being vaporware (though, Duke Nukem Forever was a thing too).
As to the project itself, I think the advertised “save = deploy with feature flag” functionality piques my interest. And I’m especially curious how it’s going to work low-level.
The “editor allows only valid syntax” feature may either be empowering, or crippling, I believe. The idea seems to come back often, but it’s main drawback IIUC is that it makes code drafting hard, interrupting the train of thoughts (at least in typical implementations). So I wonder if this challenge was taken into consideration and resolved somehow.
First of all, it needs to be noted that it’s an announcement for a yet unreleased project.
I saw a demo of this in person back in September, and at that point it was pretty far along. At least to my untrained eyes it looked fairly late stage; this is def a real thing.
They basically wrap the real Python interpreter, node interpreter, etc. in a web-based UI, and let people code on the server.
It works surprisingly well, and they have “multiplayer” concurrent editing as well.
Then supposedly you just save and open a port and you have “deployed” your server.
I would say repl.it itself is a crazy hard problem, but having to also build your own programming language and structured editor on top is even harder! That was my initial reading of Dark. But I’ll withhold judgement until I try it :)
repl.it is really cool! We feel that it’s actually simpler to build our own language and the exact experience we want than to have to fit within the bounds of existing languages. As an example, we want to meld the DB and the language to have zero impedance mismatch, which I think is something they’re not trying to do, but would also be a much bigger challenge taking their approach.
I don’t know anything about this but Paul was in my YC class, built CircleCI, and is a very smart and thoughtful guy. If anyone could do it
&c. &c. &c.
I can’t help but compare this to VB6. While I wasn’t a fan of some major parts of the VB language (error handling being the largest problem for me) I loved that once you got the IDE installed, lots of things ‘just worked’.
Are you able to draw comparisons and point to differences (in philosophy) for those who are familiar with that platform?
Yeah, I think VB6 was one of those huge shifts in computing that opened up development to so many more people, so I think there’s a lot of philosophical similarities. There’s also similarities to the rise of Java (bringing GC to the masses), Heroku (easy CRUD apps). I don’t know enough VB6 to compare effectively, but high level I agree.
Has your team done any work yet on making the IDE, particularly the structured editor, accessible to blind users? That’s the thing that always worries me about attempts to do programming without conventional text-based coding.
We haven’t yet, as we’re still figuring out what it is. As a company, we believe strongly in inclusivity, so I expect to spend time on making this work. I’ll note that code in dark is still textual, it’s just not “code is a stream of bytes in a file inode on a posix system” textual.
We’re looking for frontend and infra folks, but we all do everything and involvement in the language design is high. We’re not specifically hiring PL people as that skill is overrepresented on our team right now (and besides, PL is only a fraction of what goes into Dark).
Our goal is to make coding 100x easier, which we believe will allow a billion people to build software.
Aside from this personally being undesirable, I also think it’s unreasonable. People aren’t not programming because it’s too complex to use Emacs and the shell, or Eclipse and one of it’s many plugins. It’s because they don’t have an interest in building software. Every attempt at making programming accessible to everyone to “inspire” them to code feels misguided, because the people who want to write code will find out how and do so, they’re intrigued by the spirit of the abstract machine and the things it can do. And do 1 billion people on this planet even have the ability to program to a degree where their time wouldn’t be better spent learning a trade or an art? Maybe someone can give me a great justification for this, but Ramanujan didn’t need Mathematica to become a great mathematician, and it makes me think we’re not losing any great programmers to existing user interfaces.
Why do we need to know how to use all these tools to write software?
Aside from most of these tools not actually being mandatory, the wealth of them allows one to optimise their workflow, increase the robustness of their systems, and simplify some task. And if any of these utilities are deprecated by something better or an alternative approach appears, that link in the toolchain can be replaced without changing the rest of the programmer’s workflow. This is why we have pipes and IPC. The only complexity involved (with the concept at least) is reading the documentation of the simple tools you’re chaining together.
And do 1 billion people on this planet even have the ability to program to a degree where their time wouldn’t be better spent learning a trade or an art?
Currently, there are about 750m users of Excel. All these people are doing data editing, manipulation, and visualization, which is close to enough to coding for me. However, they lack the tools to put “code” that they write on a server to make available to others. They also lack the tools to mix the data streams that exist in their lives, whether their fitbit data, their calendars, their emails, their spotify.
As an example, think of recruiters. They do a lot of work which is automatable (scheduling, gathering interview feedback, background checks, etc), but they do it by buying one of the thousands of recruiting-automation SaaS products out there. Wouldn’t it be great if they could actually customize their own experience instead of relying on the experience that someone else created? You can extend that to data scientists, analysts, anyone in finance, and really, almost any knowledge worker at all.
Maybe we don’t need a billion people to be software engineers, but we (Dark) believe that everyone has things in their work or lives that they could automate and customize and build, if given the tools to do so.
Wow, 750M is a huge number, but it’s not implausible. I noticed the local community college teaching Excel and Word, and I’m sure every community college has similar courses. It’s easy to forget that there’s a very long tail of computer use.
Any thoughts on Google Apps Script? It’s basically JavaScript edited on the server, collaboratively, connected to the data in Google Sheets (their Excel clone). It’s a solution for sharing and editing data and code.
I’m a shell/Python/C++ person, but I used it because it was able to send mail reliably :-/ Mails sent from my hosting provider via PHP would get rejected by Google (hmmm something is fishy there …).
I can see a number of reasons why Google Apps Script isn’t the future of programming, to put it mildly. But if the goal is really to enable 750m users of excel, then starting with simple data entry and then graduating to programming seems like a good idea. And Google Sheets does let you do that.
Also I wouldn’t underestimate the power of muscle memory and all those community college courses. Google Sheets and LibreOffice still support the F2 key and simple formulas like =A1+A2 :)
Sure, there’s lots of ways to skin this cat. Our belief is that with the current ascendency of “coding” as a desirable skill, if we make something great for existing coders, the long tail (the next billion coders) will likely follow there. Of course, there are tons of companies making the opposite bet (airtable, google app script, zapier, etc).
Most people aren’t interested. Many people are who will not take up programming because of poor educational resources, its complexity, and/or some of the communities. Addressing those will get more people into programming. A few, popular languages did address one or more of those. So, that’s corroboration.
It’s more how you teach them. Most are using a subset of Python today with example problems that increase in usefulness and difficulty. They can script or do web stuff without a ton of training required. For Scheme, Racket with How to Design Programs but that’s heavier. A recent one is Pyret combined with Bootstrap program. They say it’s getting results with middle schoolers. Scratch was another one before it that made programming look like lego blocks with a community effect. Some kids even became entrepreneurs through their Scratch creations.
So, I’m mostly talking about stuff like beginner Python, Pyret w/ Bootstrap, and Scratch. Let’s them get into programming in a fun way with less, incidental complexity. What they create motives them to try the other stuff.
I agree about having one billion people programming. If that were true I wouldn’t be able to sleep at night. Re the complexity of the current server-based ecosystem, I’ll just respectfully say that liking it might be a touch of Stockholm Syndrome on our part. If we can reliably get what we need without having to deal with that complexity all the time, we should. It’s the same as the argument for serverless. It may not be for everyone but within certain problem spaces, the reduction in complexity and manual choice are worthwhile.
I don’t know that Dark is the answer, but it’s the only thing I see right now that is at least tackling the right problem.
Some choice quotes from a genius that disagrees with you
By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and in effect increases the mental power of the race.
It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them. Operations of thought are like cavalry charges in a battle — they are strictly limited in number, they require fresh horses, and must only be made at decisive moments.
Both are from Alfred Whitehead. Accessibility is always a good thing and we should strive to make programming as accessible as possible.
Your take on the matter though is second rate gatekeeping and downright regressive because you are using an appeal to some internal magical drive and talent to delineate classes of people.
Gatekeeping isn’t my intention, I’d love to help people get interested in programming. But, I think programming is already as accessible as could really be, sans towards those with certain disabilities. To design a new way that would inspire people in a way the numerous existing approaches haven’t been able to, I don’t see as anything to do with tools, but to do with culture, I guess. (Drive to program isn’t anything magical, some people are just interested in different things.)
Isn’t it a good thing that we get a lot of small moving parts that interact together instead of a single large project that covers everything?
I fear we will end up in existing, complex as a mess infrastructure and code + the new monolithic tool that just end-up as being misunderstood as a very complex web language.
plt isn’t a good tag for this, since it doesn’t actually say much if anything about programming language theory. It’s the description of a (as-yet-unreleased) commercial runtime.
I’m skeptical of the grandiose claims this project makes about making programming meaningfully more accessible to the world than it already is; and I dislike the implication that a webapp is the only sort of software that you might want to use a programming language to build.
That being said, I think that some of the ideas presented here, like trying to make calls to external HTTP APIs look identical to ordinary function calls, or making things like deploy environment and feature flags and database schemas and indices first-class language constructs, are interesting if they can be made to work. But this post isn’t going into enough detail about those neat-sounding ideas to evaluate them yet.
Gotta start somewhere, and this is our somewhere :) Ultimately, we’d love to support building all sorts of things, but this is the area we see the biggest painpoint that we think we know how to solve.
For sure. The APIs as ordinary fn calls worked ok in a naive form, but it still needs a lot of work (esp around reliability in a distributed system, but also language support for optional params and union types). The deploy thing works really great and people love it when they’re starting their app, but it needs more support for people with stable applications, esp from feature flags. With feature flags, our first two versions weren’t great, and we believe we need more tools around running both paths simultaneously, and integration with our user module, to really make it good. The DB indexing we haven’t got enough load for it to have reached the top of our priority list, but migrations are and we’re pretty confident in our solution there.
You should be skeptical…it’s marketing spam.
This was the promise of CORBA, and its abject failure taught us that this goal is neither possible nor desirable.
edit: I guess it’s possible to stake out a position of, roughly: we’ve learned a lot since CORBA, and think we can do it right this time. But I’m skeptical, and I’d like to see at least some academic response to e.g. the fallacies of distributed computing in any such claim.
CORBA said “let’s make remote objects and pretend they’re local objects”. We’re saying “there’s all these 3rdparty HTTP APIs, and they’re all vaguely RESTy, can we get a joint abstraction that incorporates retries, rate limiting, auth, network errors, etc”, and add language features that make these really easy to handle.
Most importantly, we don’t have any control of the other end of the APIs, so all we can do is make it nicer than the current (awful and disjoint) experience.
First of all, it needs to be noted that it’s an announcement for a yet unreleased project.
That said, IIUC the author created CircleCI (?), which would lend him at least a bit more initial credibility than in case of some Joe Random Hacker in a similar situation. Some history of successful delivery reduces the chances of this being vaporware (though, Duke Nukem Forever was a thing too).
As to the project itself, I think the advertised “save = deploy with feature flag” functionality piques my interest. And I’m especially curious how it’s going to work low-level.
The “editor allows only valid syntax” feature may either be empowering, or crippling, I believe. The idea seems to come back often, but it’s main drawback IIUC is that it makes code drafting hard, interrupting the train of thoughts (at least in typical implementations). So I wonder if this challenge was taken into consideration and resolved somehow.
I saw a demo of this in person back in September, and at that point it was pretty far along. At least to my untrained eyes it looked fairly late stage; this is def a real thing.
FWIW I’ve been following repl.it, and it seems like their vision is similar, except that they don’t have their own language:
https://repl.it/site/blog/deploy
They basically wrap the real Python interpreter, node interpreter, etc. in a web-based UI, and let people code on the server.
It works surprisingly well, and they have “multiplayer” concurrent editing as well.
Then supposedly you just save and open a port and you have “deployed” your server.
I would say repl.it itself is a crazy hard problem, but having to also build your own programming language and structured editor on top is even harder! That was my initial reading of Dark. But I’ll withhold judgement until I try it :)
repl.it is really cool! We feel that it’s actually simpler to build our own language and the exact experience we want than to have to fit within the bounds of existing languages. As an example, we want to meld the DB and the language to have zero impedance mismatch, which I think is something they’re not trying to do, but would also be a much bigger challenge taking their approach.
I don’t know anything about this but Paul was in my YC class, built CircleCI, and is a very smart and thoughtful guy. If anyone could do it &c. &c. &c.
Author here, thanks for posting. Happy to answer questions!
I can’t help but compare this to VB6. While I wasn’t a fan of some major parts of the VB language (error handling being the largest problem for me) I loved that once you got the IDE installed, lots of things ‘just worked’.
Are you able to draw comparisons and point to differences (in philosophy) for those who are familiar with that platform?
Yeah, I think VB6 was one of those huge shifts in computing that opened up development to so many more people, so I think there’s a lot of philosophical similarities. There’s also similarities to the rise of Java (bringing GC to the masses), Heroku (easy CRUD apps). I don’t know enough VB6 to compare effectively, but high level I agree.
One of my Unpopular Programming Opinions is that VB6 is still way ahead of its time
It absolutely is. This is how apps should be made. It is actually fun to design a UI this way.
Has your team done any work yet on making the IDE, particularly the structured editor, accessible to blind users? That’s the thing that always worries me about attempts to do programming without conventional text-based coding.
We haven’t yet, as we’re still figuring out what it is. As a company, we believe strongly in inclusivity, so I expect to spend time on making this work. I’ll note that code in dark is still textual, it’s just not “code is a stream of bytes in a file inode on a posix system” textual.
Are you looking for any more language hackers right now?
Just posted today: https://news.ycombinator.com/item?id=19283993
We’re looking for frontend and infra folks, but we all do everything and involvement in the language design is high. We’re not specifically hiring PL people as that skill is overrepresented on our team right now (and besides, PL is only a fraction of what goes into Dark).
dark is just a project as far as i can see
Reminded me of darkbasic, they had a java like language in the past too iirc:
https://en.wikipedia.org/wiki/The_Game_Creators
Situation: there are too many competing technologies for building the web…
Situation: there are not any real contenders in orchestrating those technologies in building software
Aside from this personally being undesirable, I also think it’s unreasonable. People aren’t not programming because it’s too complex to use Emacs and the shell, or Eclipse and one of it’s many plugins. It’s because they don’t have an interest in building software. Every attempt at making programming accessible to everyone to “inspire” them to code feels misguided, because the people who want to write code will find out how and do so, they’re intrigued by the spirit of the abstract machine and the things it can do. And do 1 billion people on this planet even have the ability to program to a degree where their time wouldn’t be better spent learning a trade or an art? Maybe someone can give me a great justification for this, but Ramanujan didn’t need Mathematica to become a great mathematician, and it makes me think we’re not losing any great programmers to existing user interfaces.
Aside from most of these tools not actually being mandatory, the wealth of them allows one to optimise their workflow, increase the robustness of their systems, and simplify some task. And if any of these utilities are deprecated by something better or an alternative approach appears, that link in the toolchain can be replaced without changing the rest of the programmer’s workflow. This is why we have pipes and IPC. The only complexity involved (with the concept at least) is reading the documentation of the simple tools you’re chaining together.
Currently, there are about 750m users of Excel. All these people are doing data editing, manipulation, and visualization, which is close to enough to coding for me. However, they lack the tools to put “code” that they write on a server to make available to others. They also lack the tools to mix the data streams that exist in their lives, whether their fitbit data, their calendars, their emails, their spotify.
As an example, think of recruiters. They do a lot of work which is automatable (scheduling, gathering interview feedback, background checks, etc), but they do it by buying one of the thousands of recruiting-automation SaaS products out there. Wouldn’t it be great if they could actually customize their own experience instead of relying on the experience that someone else created? You can extend that to data scientists, analysts, anyone in finance, and really, almost any knowledge worker at all.
Maybe we don’t need a billion people to be software engineers, but we (Dark) believe that everyone has things in their work or lives that they could automate and customize and build, if given the tools to do so.
Also I’m curious how Dark compares to ifttt or Zapier? I think they are addressing these kinds of use cases.
Yes, we think of ourselves as a mix of Zapier, Glitch, Lambda, and Parse.
Wow, 750M is a huge number, but it’s not implausible. I noticed the local community college teaching Excel and Word, and I’m sure every community college has similar courses. It’s easy to forget that there’s a very long tail of computer use.
Any thoughts on Google Apps Script? It’s basically JavaScript edited on the server, collaboratively, connected to the data in Google Sheets (their Excel clone). It’s a solution for sharing and editing data and code.
I’m a shell/Python/C++ person, but I used it because it was able to send mail reliably :-/ Mails sent from my hosting provider via PHP would get rejected by Google (hmmm something is fishy there …).
I can see a number of reasons why Google Apps Script isn’t the future of programming, to put it mildly. But if the goal is really to enable 750m users of excel, then starting with simple data entry and then graduating to programming seems like a good idea. And Google Sheets does let you do that.
Also I wouldn’t underestimate the power of muscle memory and all those community college courses. Google Sheets and LibreOffice still support the F2 key and simple formulas like =A1+A2 :)
Sure, there’s lots of ways to skin this cat. Our belief is that with the current ascendency of “coding” as a desirable skill, if we make something great for existing coders, the long tail (the next billion coders) will likely follow there. Of course, there are tons of companies making the opposite bet (airtable, google app script, zapier, etc).
Most people aren’t interested. Many people are who will not take up programming because of poor educational resources, its complexity, and/or some of the communities. Addressing those will get more people into programming. A few, popular languages did address one or more of those. So, that’s corroboration.
What kind of programming languages address those sorts of complaints? Just as an example.
It’s more how you teach them. Most are using a subset of Python today with example problems that increase in usefulness and difficulty. They can script or do web stuff without a ton of training required. For Scheme, Racket with How to Design Programs but that’s heavier. A recent one is Pyret combined with Bootstrap program. They say it’s getting results with middle schoolers. Scratch was another one before it that made programming look like lego blocks with a community effect. Some kids even became entrepreneurs through their Scratch creations.
So, I’m mostly talking about stuff like beginner Python, Pyret w/ Bootstrap, and Scratch. Let’s them get into programming in a fun way with less, incidental complexity. What they create motives them to try the other stuff.
I agree about having one billion people programming. If that were true I wouldn’t be able to sleep at night. Re the complexity of the current server-based ecosystem, I’ll just respectfully say that liking it might be a touch of Stockholm Syndrome on our part. If we can reliably get what we need without having to deal with that complexity all the time, we should. It’s the same as the argument for serverless. It may not be for everyone but within certain problem spaces, the reduction in complexity and manual choice are worthwhile.
I don’t know that Dark is the answer, but it’s the only thing I see right now that is at least tackling the right problem.
Only thing?!
Some choice quotes from a genius that disagrees with you
Both are from Alfred Whitehead. Accessibility is always a good thing and we should strive to make programming as accessible as possible.
Your take on the matter though is second rate gatekeeping and downright regressive because you are using an appeal to some internal magical drive and talent to delineate classes of people.
Gatekeeping isn’t my intention, I’d love to help people get interested in programming. But, I think programming is already as accessible as could really be, sans towards those with certain disabilities. To design a new way that would inspire people in a way the numerous existing approaches haven’t been able to, I don’t see as anything to do with tools, but to do with culture, I guess. (Drive to program isn’t anything magical, some people are just interested in different things.)
Then I misinterpreted what you said and the disagreement seems to be around
I don’t think this is true.
I explained myself poorly, it’s no surprise you misinterpreted it, my apologies
Isn’t it a good thing that we get a lot of small moving parts that interact together instead of a single large project that covers everything?
I fear we will end up in existing, complex as a mess infrastructure and code + the new monolithic tool that just end-up as being misunderstood as a very complex web language.
pltisn’t a good tag for this, since it doesn’t actually say much if anything about programming language theory. It’s the description of a (as-yet-unreleased) commercial runtime.