I share the author’s frustrations, but I doubt the prescriptions as presented will make a big difference, partly because they have been tried before.
And they came up with Common Lisp. And it’s huge. The INCITS 226–1994 standard consists of 1153 pages. This was only beaten by C++ ISO/IEC 14882:2011 standard with 1338 pages some 17 years after. C++ has to drag a bag of heritage though, it was not always that big. Common Lisp was created huge from the scratch.
This is categorically untrue. Common Lisp was born out of MacLisp and its dialects, it was not created from scratch. There was an awful lot of prior art.
This gets at the fatal flaw of the post: not addressing the origins of the parts of programming languages the author is rejecting. Symbolic representation is mostly a rejection of verbosity, especially of that in COBOL (ever try to actually read COBOL code? I find it very easy to get lost in the wording) and to more closely represent the domains targetted by the languages. Native types end up existing because there comes a time where the ideal of maths meets the reality of engineering.
Unfortunately, if you write code for other people to understand, you have to teach them your language along with the code.
I don’t get this criticism of metaprogramming since it is true of every language in existence. If you do metaprogramming well, you don’t have to teach people much of anything. In fact, it’s the programmer that has to do the work of learning the language, not the other way around.
The author conveniently glosses over the fact that part of the reason there are so many programming languages is that there are so many ways to express things. I don’t want to dissuade the author from writing or improving on COBOL to make it suitable for the 21st century; they can even help out with the existing modernization efforts (see OO COBOL), although they may be disappointed to find out COBOL is not really that small.
If you do click through and finish the entire post you’ll see the author isn’t really pushing for COBOL. The key point is made: “Aren’t we unhappy with the environment in general?” This, I agree, is the main problem. No solution is offered, but there is a decent sentiment about responsibility.
Also if you want a smaller Lisp than CL with many of it’s more powerful features, there’s always ISLisp, which is one of the more under-appreciated languages I’ve seen. It has many of the nicer areas of CL, with the same syntax (unlike Dylan which switched to a more Algol-like), but still has a decent specification weighing in at a mere 134 pages.
For me, the One Thing missing from python is a rigorous static typechecker. I miss that a lot when writing python code. That said, I do generally enjoy working in the language.
I share the feelings. I’d add that the common functions that you might expect like map/filter/sort/… have sometimes weird names or syntax and reminds me PHP…
I never understood this complaint/desire to add static typing to languages that are completely antithetical to the idea. Not every feature that exists in other languages should be added to every language (that leads to C++, Rust and PHP). Python is a dynamic language and it doesn’t make sense to bolt rigorous static typing on top, and it’s never going to be completely sound, because of its inherent dynamic nature. Why not use a language with a sound strict typing system, like OCaml or Haskell or Elm or what have you?
Actually, I’d even be happier if Python didn’t include lambdas, functools and such. Having no support for functional programming would be way less frustrating than having half-assed support for functional programming, especially given that the community frowns upon their use anyway!
I strongly prefer OCaml, Haskell, Elm, etc. But those languages don’t have anywhere near the community buy-in as Python, so there’s much more friction in using them for a project (especially a collaborative one).
Bananas for data types; no facilities for arrays (without extensions like numpy)
Crap “error” handling: Dynamic unwind is the last thing we want for error handling, but
1990’s module system aka PYTHONPATH means everyone invents their own plugin system. Hooray for virtualenv, you’ve just traded one problem for two.
Zero legacy support. Python applications are expected to either be sealed (and risk platform bugs, of which there have been many) or be rewritten (often substantially) every time there’s a new release. Hopefully this is changing – Python3 has settled down considerably, but I need another ten years.
I could go on. Really, Python is one of the worst successful languages I’ve ever seen or used. and I’m so glad other languages are eating its lunch (go, rust, cobol) – even though I don’t particularly rate those languages (although admittedly for other reasons).
Bananas for data types; no facilities for arrays (without extensions like numpy)
I don’t really think it’s a big deal to need extensions for data types, if it wasn’t for Python’s “batteries included” mantra. It’s more like “a few very specific batteries of varying quality are included”.
Crap “error” handling: Dynamic unwind is the last thing we want for error handling
Could you unpack that a bit? I’m genuinely interested to know what is wrong with its exception handling and what alternatives are better.
1990’s module system aka PYTHONPATH means everyone invents their own plugin system. Hooray for virtualenv, you’ve just traded one problem for two.
Yeah, modules in Python are a bit fiddly.
Zero legacy support.
I can imagine that’s annoying when working on an old application, but I do think improvements need to happen, and sometimes the best way forward is to do an overhaul instead of lugging around broken stuff forever. In this, Python is a bit of a victim of its own success; the reason it was so hard to switch to Python 3 was mostly in the 3rd party libraries. This is tricky to get right (but it can be worse; look at PHP with its ancient legacy cruft sticking around and continually breaking very large things on each major version), and I really wonder how the new languages like Go and Rust fare with legacy support. They haven’t been around for as long as Python.
Bananas for data types; no facilities for arrays (without extensions like numpy)
I don’t really think it’s a big deal to need extensions for data types, if it wasn’t for Python’s “batteries included” mantra. It’s more like “a few very specific batteries of varying quality are included”.
A really useful data type is the “table”, but it’s extremely difficult to write one in Python, so nobody bothers. Everyone just writes:
for row in q.fetchall():
all day, and think this is the best they can do. But “my database” knows what a table is. Its tables are pretty good. Why can’t I implement a table type that’s worth half that? Why can’t I even approach sqlite-level of functionality? If Python is so great, why is something that everyone uses every day nearly impossible to implement in Python?
Another really useful data type is a “date”. Dates are also hard to implement in Python, but this one people have tried. Unfortunately, the one most popular had a bug involving DST east of GMT creating a lot of weird problems for people. Dates have a continuous nature that the integer usually used to represent them doesn’t have, which also causes strangeness for people. Intervals are really similar to dates – you can add them, but intervals don’t have timezones. Dates do. This causes more bugs. And so on.
Getting batteries included with your golf cart is stupid because you’re still driving a golf cart.
Crap “error” handling: Dynamic unwind is the last thing we want for error handling
Could you unpack that a bit? I’m genuinely interested to know what is wrong with its exception handling and what alternatives are better.
Common Lisp has a “condition system” which is better.
To separate the language-specific element: Conditions can be implemented in Python trivially. Most of the issue with error handling is cultural, and people continue to argue between “exceptions” (e.g. longjmp, dynamic-unwind, nonlocal goto, etc), and “error codes” – like the issue is merely signalling exceptional state one way or another or something like that.
To see what happens when you think that the point of errors is to tell people about them, look at the innovation in this space you’re probably much more familiar with: So-called “railway-oriented programming” or A+ Promises. They’re the same thing. Stack unwinds, but we never rewind.
That’s the trick: We don’t just want to say “help, I can’t proceed any further”, we want to say “I can’t proceed any further unless…” – and that’s exactly what conditions give you.
If you want to imagine what this could have looked like in Python, imagine instead of an except IOError, e: you could say when IOError, e: do_something() and then do_something() could return so that whatever was trying to write to the disk (or whatever) could retry. You could implement this with functions and while loops, but that would never fix all the other libraries that are still wrong.
but it can be worse; look at PHP with its ancient legacy cruft sticking around and continually breaking very large things on each major version
Oh indeed. I’m aware things can always be worse. But if I’m going to fantasise about how things are going to be better, I’m going to shoot a hell of a lot higher than PHP.
A really useful data type is the “table”, but it’s extremely difficult to write one in Python, so nobody bothers.
What exactly would you expect from a table type that’s better than the “for row in fetchall()” style functionality? The database engines I know about will literally return a list of tuples to the client. Sqlite seems to allow stepping through this list to return the tuple at the current point a la “cursors”. Or is it more the row-number/column-name type access you’re aiming at here? That’s pretty trivial to implement with “magic methods”. For some weird reason that’s not the default behaviour in psycopg2 but you can ask for a dictionary-like cursor when making the db connection.
Common Lisp has a “condition system” which is better.
Ah, I see what you mean now. The distinction is about continuable versus non-continuable conditions/exceptions. That reminds me, I’d like to make continuable exceptions easier to handle in CHICKEN, as currently they’re a bit awkward to deal with right now.
But if I’m going to fantasise about how things are going to be better, I’m going to shoot a hell of a lot higher than PHP.
What exactly would you expect from a table type that’s better than the “for row in fetchall()” style functionality?
Think about what you’re doing inside those for-loops and how similar many patterns you might have there. Some of the ones I see every day:
Reorder (or build a permutation array) the table by one, two, many compound columns. In k, all vector operators are extended to tables by having tables implemented as a list of columns; arrays all the same length.
Join/zip two tables together (exceptionally common). Gross in Python, but in k this is just ,
Determine if some table is inside another table.
Group some table rows by some columns.
Use a column as a rule on other columns. For example, if I want a table of switches, and one column to specify which switch to flip, in q this is
@[t;t`rules;@[;;not];til count t]
In Python, you need, if row-ordered something like:
for x in t:
x[x["rules"]] ^= x[x["rules"]]
or if column-ordered:
for i,x in enumerate(t["rules"]):
t[x][i] ^= t[x][i]
Neither is great, and I’ve seen worse contortions trying to rewrite it in SQL…
Deleting rows from a table (usually simulated with list.filter, but this conses a lot)
Upserting (often simulated with an index)
And so on.
The database engines I know about will literally return a list of tuples to the client
One way to think about this: The “database engine” is just another programming language, and it’s another one that’s better than Python at tables. The challenge is to have tables in not a database engine.
It’s funny, because as soon as you put tables in there, plus “a little bit more”, you have a “database engine”, but exactly how much “a little bit more” is, and what you get for it makes you wonder why you just didn’t do it in Python (or C or Rust or Go whatever) to begin with.
Back in the day, the reason “the database” was separate was because a separate team managed it, but I feel like we’re past this now. Keeping tables out of languages isn’t some layer-violation, it’s a space of real problems people are working on every day that language designers are just ignoring for inertia reasons.
Ah, I see what you mean now. The distinction is about continuable versus non-continuable conditions/exceptions.
Well it’s one solution. I’m not closed to others, but it’s an obvious one that’s “better” by some definition. It’s worse by another (longer code) which might also be important: I put a fair amount of effort into fixing ECL’s file/IO “exceptions” to be restartable, but I don’t think anyone ever used them but me. Lisp is great, but people keep writing Java in it…
I’m still thinking about how language can help with this.
Think about what you’re doing inside those for-loops and how similar many patterns you might have there.
Hm, that’s thought-provoking, thank you for that! I see what you mean now, never really considered it to be a huge problem, but these patterns do occur over and over.
It’s funny, because as soon as you put tables in there, plus “a little bit more”, you have a “database engine”, but exactly how much “a little bit more” is, and what you get for it makes you wonder why you just didn’t do it in Python (or C or Rust or Go whatever) to begin with.
In most cases I would argue you didn’t write the right query to return just the data you need in the right form. Nine times out of ten you see people writing horribly contorted code that’s dog slow in their programming language (often performing tons of extremely dumb queries) to massage data into shape, in a way that could be done much better and more efficiently with SQL (often with a single query instead of many). So even if you have the right tools people will abuse them.
Back in the day, the reason “the database” was separate was because a separate team managed it, but I feel like we’re past this now.
I would tend to disagree: the most important part (for me) about a database are the ACID properties offered by RDBMSes. It’s a separation of concerns; you don’t want your language to implement a storage engine, just like you don’t want your language to implement a raw file system; leave that to the OS kernel. You could do both in a library of course, but I’d much prefer a specialized separate tool to do it. Nevertheless, having good direct support for “tables” like you mention would be nice and a good and natural extension for dealing with result sets or tabular data that doesn’t need to go to an external database.
I put a fair amount of effort into fixing ECL’s file/IO “exceptions” to be restartable, but I don’t think anyone ever used them but me.
My hat’s off to you! We need more people to put effort into making existing good tools even better.
In most cases I would argue you didn’t write the right query to return just the data you need in the right form. Nine times out of ten you see people writing horribly contorted code that’s dog slow in their programming language (often performing tons of extremely dumb queries) to massage data into shape, in a way that could be done much better and more efficiently with SQL (often with a single query instead of many). So even if you have the right tools people will abuse them.
Yes. That happens very often. But I want to stress that tables don’t just live in the database.
I have an ad server that can either retrieve ads from its local database, or from a remote server (using HTTP). The publisher (client) requests ads from me, and I’ll basically be an aggregator. If the publisher asks for 10 ads, and the advertiser gives me 3, I might ask for 7 from another advertiser and glue them together.
The ad response is a table - each row representing a different ad, and each column representing some attribute (like the title, description, click-URL). I’m doing asof join (basically a merge-sort) to glue these together since I know the client may only take the first few ads so everyone who does this puts the higher-value ads at the top.
I’m given search keywords, so I do a query on this table – it’s a virtual table really, since it spans multiple databases (local and remote) but you can see this is clearly a table lookup made difficult by committee.
Knowing that, look at the schema Yahoo uses (starts around page 87). Do you see that table? How about when Google does it? Do you see the table now?
the most important part (for me) about a database are the ACID properties offered by RDBMSes
As soon as I discovered logging, I realised I never needed ACID Again. I rarely use an ACID database these days, and my application design has become a lot simpler as a result.
I remember I noticed that many business applications want their database to remember all the changes. And who made them. So many CRUD applications will in addition to all the insert/delete/update operations, duplicate those into an “audit” table of some kind. There may be even some business processes for recovering/rollback/cherry-picking values from that table.
However another way you can do this is to imagine you have a single “changes” table. Inserts to it are cheap! it’s just file append! Then all of your queries go to materialised views (re)built from those changes. In a catastrophic situation it’s trivial to replay the log and get the views back. Or change the schema and replay so you get your data right. No migrations.
It’s like patch theory for databases.
Here you can see we have atomicity when we want it (simply block queries to the new views while you’re building them), consistency (all the new views are built together), integrity (duh), and durability. In fact, this approach is so easy, you’re likely to get it right the first time. Putting b-trees on disks on the other hand is what all the “database engines” do, and to get acceptable performance means handling a lot of edge cases. That makes a lot of code- places for bugs to hide.
It’s a separation of concerns; you don’t want your language to implement a storage engine
I’m not so sure. Being able to pickle/map disk-objects to values is incredibly valuable so why not a table?
k/q supports serialising all data types (including lambdas, btw) so they can be sent over the network or written to the disk. On the disk, a table can become a directory where each column is a separate file. mmap() brings them into memory and their layout is similar enough to their purely-in-memory version that few operations need to know the difference. This feels like an extreme version of what I used to get out of hibernation/pickling, and yet it’s so simple! And look at all I get!
There’s some support for metaprogramming in the form of metaclasses, and you can do lots of magic with magic methods, iterators and context managers. There are even hooks that you can call when a class instantiates from an abstract base class. It simply isn’t part of the culture to do lots of metaprogramming, as magic is frowned upon as not being the “one obvious way to do it”. But there are projects that rely heavily on this stuff. For example, the lxml builder uses magic methods a lot, and Django uses model introspection to autogenerate database migrations.
I respect the attitude of Python culture of insisting on simple code (and to be fair, Python code tends to be very readable by default), but it’s not my favourite language. Python is relentlessly imperative and the functional style is really frowned upon, it has no macros (the existing metaprogramming is too clumsy to express certain things) and whitespace indentation makes working with the REPL a total pain in the neck.
I completely forgot about metaclasses, thanks for mentioning it. I actually learnt the basics of FP with Python 1.5 and the lambda/map/filter/reduce which at the time was already “ok, but don’t use it”. I continued to use it sporadically but found that list comprehension actually solved a lot of the use cases where I was using filter/map/reduce (although reduce in list comprehension is a bit of a headache). It’s weird, because I wouldn’t say Python is that imperative from a feature standpoint given that you can pass around functions, methods and classes (and now, even type descriptions), but seems to be, as you pointed it out, more culturally imperative. I wonder why that is, do you know happen to know the reasons behind this attitude/perception?
It seems Guido believes functional programming is hard to understand. Furthermore, it doesn’t fit his idea of Python, see this old Slashdot post (search for “functional programming”).
You might be aware, but Red is following that path. But they’ve gone off on a cryptocurrency tangent; I’m not quite sure what’s going on there anymore.
I think dialecting ala Rebol is super interesting, but I also think this sort of “wordy” input like AppleScript and DCL will eventually just become short forms that often require just as much effort to read later… that’s how you’d have things like show device... foreshortened to sho dev ....
Having said that, SRFI-10 or #. form from CommonLisp is a happy medium, I think.
hahahaha oh lord, I know what you mean. I still have ancient devices burned in my brain as well, like OpenVMS and what not. Still, I think it goes to show that making things more “natural language-like” doesn’t really mean we want to write like that… there’s probably some balance to be struck between succinctness and power that we haven’t figured out yet
I also loved the bit of engagement at the end with the buttons. Been a string of really well written (light) technical articles lately, hope the trend continues.
I ported a REBOL app (using the full paid stack) to C# – the code inflation and challenge of making a 1:1 exact copy (no retraining port) was phenomenal. Most stuff took nearly an order of magnitude more code. There were some wins (dynamic layouts, resizing, performance) – but REBOL had shockingly good bang for the buck and dialects only really took a few days to grok.
I think Forth is really the language the 21st century needed. I see Forth as having little-to-no syntax, which gives it a wealth of expressiveness. Every Forth program is practically a DSL, or it isn’t, if you don’t want to write it like that. It’s typeless (though some implementations provide typing). Postfix is powerful. Any Forth library can expose a simple wordlist intended as the user API, and thanks to the hyperstatic environment avoid name conflicts with user code. Any data structures you like. It’s procedural, functional, stateful, stateless, dataflow-y, logical - whatever you want. Access to the return stack is equally as useful; mega fast (just skip all the cruft), RDROP RECURSE, keeping arbitrary data out of the way for efficient stack usage. Some forths implement scoped variables if you want them (I know gforth does). I just think the main problem is the bickering community that doesn’t get much done, and the millions of implementations but lack of actual programs. The history of Forth is one of not adapting and not evolving. It’s more of a guideline, less a specific language, and that’s key to what Chuck Moore espoused it to be in his book even if the book is practically a tutorial on it’s implementationProgramming a Problem Oriented Language (https://colorforth.github.io/POL.htm). Maybe it’s too esoteric. I bet it would’ve blown up if someone had implemented tk bindings. Maybe someone should release a pre-installed Forth arduino. I really lost track of what I was going to say here, but I think some kind of point can be derived. Use Forth? Try it out at least.
The author’s first and third points kind of contradict each other.
The whole point of creating a DSL is to make a language that mimics the end user’s domain. It’s analogous to how the English sentence syntax he recommends mimics a regular language. If it takes a lot of mental effort to learn a new DSL then it’s a bad DSL.
I also disagree with shrugging off performance concerns. Look at GMail’s new UI rollout - even in a high level web app, users still care about performance.
I also disagree with shrugging off performance concerns. Look at GMail’s new UI rollout - even in a high level web app, users still care about performance.
Sure, but GMail’s UI isn’t slow because it’s written in JavaScript. It’s slow because Google doesn’t give a fuck about making it fast.
COBOL has different semantics for ADD, ADD TO, ADD TO GIVING, and COMPUTE, as well as DIVIDE, DIVIDE INTO, and DIVIDE BY. I don’t think COBOL is the technical strawman he’s looking for, which undermines the (arguably valid) social point he’s trying to make.
Yup. He’s made the same mistake that informed a lot of ‘english-like’ languages & confused ‘simple’ with ‘already familiar’, ignoring what needs to be unlearned to make coding possible. (How much easier would learning SQL be if the constructs didn’t resemble common english phrases & we therefore didn’t immediately expect to be able to reorder them?)
I’m thinking of ruby when reading this. If you remember _why’s guides, he was very enthusiastic about the natural language part of Ruby, and for me that was one of the things I liked about it. However, where it was easy to create API’s that felt like natural language, it also gave power to some syntactic nightmares. If you don’t believe me, just take a look at all Ruby submissions to the Advent of Code challenges, and you’ll find neat little one-liners that are impossible to parse by head.
I think the biggest issue these days is language longevity.
Creating a language and its surrounding ecosystem takes decades of work – and it’s usually ruined by people who just want to add yet-another language feature, until the language collapses under its own weight. Then the cycle repeats, with decades of effort wasted.
See Rust, C#, Scala, C++ … for the various stages of this.
There really needs to be some soul-searching/research on preventing this from happening over and over.
Readability is not code for “I can decode it with stackoverflow or google”
It doesn’t mean “it looks like every other language” either; That’s Approachability, and it may imply a level of Readability, it isn’t the same thing either.
We programmers can change ourselves to learn to Read new languages, and with practice and experience, can be many many times more successful* with “unreadable” languages than the Readable ones.
* That’s success as measured by personal metrics (salary, job satisfaction) and business metrics (time-to-deliver, cost of sales, risk), and not e.g. popularity.
It’s naïve to think that the language is responsible for the quality of code and that by adding some bells and whistles (or removing some bells and whistles), we can automatically make everything better. We were not happy with Fortran and COBOL, so we invented C++ and Java only to be unhappy with them too in some 20–30 years.
The structured programming debate when they abolished goto. It was a pretty good improvement to the languages overall. It was only spoiled by the fact that “practical” people didn’t understand shit. The point was to make the code easier to understand by giving it regular structure. The complex structures could have been conveyed by tail recursion.
Well people found optimal tail recursion hard to implement, so instead they introduced ways to violate the rules that made structured programming work in the first place, the idea that you cannot haphazardly jump around in the code was weakened. They introduced break/continue into loops, and of course, the return statement. Well of course the structured programming was not as useful afterwards.
Likewise the object oriented programming was about processes and messages. Very important ideas to control for concurrent programming. People diluted it into a dogma that dominated the industry for few years until few figured they had chased red herrings there.
Of course if you fuck up with the theory part of your work, you’re certified to come up with tools that aren’t better than the things they replace. Jonathan Blow didn’t get this, Go completely missed it, Rust strike force almost figured it out. Whole C++ committee is out on the field and didn’t get the memo. Surprisingly it’s hard to grok that the theory is important.
So if I were to invent a programming language for the 21st century, I would reinvent being responsible instead. I would reinvent learning your tools; I would reinvent being attentive to essential details and being merciless to accidental complexity.
If you’re going to try this the same way that you try to reinvent programming languages, the outcome won’t be different. For example of this you can just look at Bret Victor’s stuff or the EVE text editor. Also you’re going need a better language to come up with better tools. The language is your primary tool for being attentive on the detail and explaining complexity.
Languages are tools. When I want to use a knife, I prefer a sharper, more durable, and more comfortable to use knife. Especially if I’m to use it ~8h each workday. Sure, I can probably do similar things with a blunt knife, but a better tool allows me to do more and faster. Unless I misunderstood something about the article, that would seem to be my answer to it - I don’t seem to see the point in the complaints. I know my knifes and there are a lot of interesting new ideas how better ones could be made. Rust, Luna (!), Elm, Zig, there are a lot of interesting new experiments that may (or already do) bring new quality to the table. There are also old knifes which sometimes get refurbished and strenghtened, improved and reinvigorated, and slowly transformed into something completely new, like OCaml/Reason, Red, Clojure, just from the top of my head. As to the general idea of “we should do better” - sure, but isn’t all life more or less about it? And better tools can help with this in various ways.
I share the author’s frustrations, but I doubt the prescriptions as presented will make a big difference, partly because they have been tried before.
This is categorically untrue. Common Lisp was born out of MacLisp and its dialects, it was not created from scratch. There was an awful lot of prior art.
This gets at the fatal flaw of the post: not addressing the origins of the parts of programming languages the author is rejecting. Symbolic representation is mostly a rejection of verbosity, especially of that in COBOL (ever try to actually read COBOL code? I find it very easy to get lost in the wording) and to more closely represent the domains targetted by the languages. Native types end up existing because there comes a time where the ideal of maths meets the reality of engineering.
I don’t get this criticism of metaprogramming since it is true of every language in existence. If you do metaprogramming well, you don’t have to teach people much of anything. In fact, it’s the programmer that has to do the work of learning the language, not the other way around.
The author conveniently glosses over the fact that part of the reason there are so many programming languages is that there are so many ways to express things. I don’t want to dissuade the author from writing or improving on COBOL to make it suitable for the 21st century; they can even help out with the existing modernization efforts (see OO COBOL), although they may be disappointed to find out COBOL is not really that small.
If you do click through and finish the entire post you’ll see the author isn’t really pushing for COBOL. The key point is made: “Aren’t we unhappy with the environment in general?” This, I agree, is the main problem. No solution is offered, but there is a decent sentiment about responsibility.
Also if you want a smaller Lisp than CL with many of it’s more powerful features, there’s always ISLisp, which is one of the more under-appreciated languages I’ve seen. It has many of the nicer areas of CL, with the same syntax (unlike Dylan which switched to a more Algol-like), but still has a decent specification weighing in at a mere 134 pages.
At the end I was looking for Python but it wasn’t even listed. :(
Python is…
It’s only downside (in my opinion and limited experience) is that it usually will be slower than something comparable in a compiled language.
For me, the One Thing missing from python is a rigorous static typechecker. I miss that a lot when writing python code. That said, I do generally enjoy working in the language.
I share the feelings. I’d add that the common functions that you might expect like map/filter/sort/… have sometimes weird names or syntax and reminds me PHP…
I never understood this complaint/desire to add static typing to languages that are completely antithetical to the idea. Not every feature that exists in other languages should be added to every language (that leads to C++, Rust and PHP). Python is a dynamic language and it doesn’t make sense to bolt rigorous static typing on top, and it’s never going to be completely sound, because of its inherent dynamic nature. Why not use a language with a sound strict typing system, like OCaml or Haskell or Elm or what have you?
Actually, I’d even be happier if Python didn’t include lambdas, functools and such. Having no support for functional programming would be way less frustrating than having half-assed support for functional programming, especially given that the community frowns upon their use anyway!
I strongly prefer OCaml, Haskell, Elm, etc. But those languages don’t have anywhere near the community buy-in as Python, so there’s much more friction in using them for a project (especially a collaborative one).
Did you checkout mypy and the typing module? I had the same feeling, and am super happy with both.
It has many other downsides.
PYTHONPATHmeans everyone invents their own plugin system. Hooray for virtualenv, you’ve just traded one problem for two.I could go on. Really, Python is one of the worst successful languages I’ve ever seen or used. and I’m so glad other languages are eating its lunch (go, rust, cobol) – even though I don’t particularly rate those languages (although admittedly for other reasons).
I don’t really think it’s a big deal to need extensions for data types, if it wasn’t for Python’s “batteries included” mantra. It’s more like “a few very specific batteries of varying quality are included”.
Could you unpack that a bit? I’m genuinely interested to know what is wrong with its exception handling and what alternatives are better.
Yeah, modules in Python are a bit fiddly.
I can imagine that’s annoying when working on an old application, but I do think improvements need to happen, and sometimes the best way forward is to do an overhaul instead of lugging around broken stuff forever. In this, Python is a bit of a victim of its own success; the reason it was so hard to switch to Python 3 was mostly in the 3rd party libraries. This is tricky to get right (but it can be worse; look at PHP with its ancient legacy cruft sticking around and continually breaking very large things on each major version), and I really wonder how the new languages like Go and Rust fare with legacy support. They haven’t been around for as long as Python.
A really useful data type is the “table”, but it’s extremely difficult to write one in Python, so nobody bothers. Everyone just writes:
all day, and think this is the best they can do. But “my database” knows what a table is. Its tables are pretty good. Why can’t I implement a table type that’s worth half that? Why can’t I even approach sqlite-level of functionality? If Python is so great, why is something that everyone uses every day nearly impossible to implement in Python?
Another really useful data type is a “date”. Dates are also hard to implement in Python, but this one people have tried. Unfortunately, the one most popular had a bug involving DST east of GMT creating a lot of weird problems for people. Dates have a continuous nature that the integer usually used to represent them doesn’t have, which also causes strangeness for people. Intervals are really similar to dates – you can add them, but intervals don’t have timezones. Dates do. This causes more bugs. And so on.
Getting batteries included with your golf cart is stupid because you’re still driving a golf cart.
Common Lisp has a “condition system” which is better.
To separate the language-specific element: Conditions can be implemented in Python trivially. Most of the issue with error handling is cultural, and people continue to argue between “exceptions” (e.g. longjmp, dynamic-unwind, nonlocal goto, etc), and “error codes” – like the issue is merely signalling exceptional state one way or another or something like that.
To see what happens when you think that the point of errors is to tell people about them, look at the innovation in this space you’re probably much more familiar with: So-called “railway-oriented programming” or A+ Promises. They’re the same thing. Stack unwinds, but we never rewind.
That’s the trick: We don’t just want to say “help, I can’t proceed any further”, we want to say “I can’t proceed any further unless…” – and that’s exactly what conditions give you.
If you want to imagine what this could have looked like in Python, imagine instead of an
except IOError, e:you could saywhen IOError, e: do_something()and thendo_something()could return so that whatever was trying to write to the disk (or whatever) could retry. You could implement this with functions and while loops, but that would never fix all the other libraries that are still wrong.Oh indeed. I’m aware things can always be worse. But if I’m going to fantasise about how things are going to be better, I’m going to shoot a hell of a lot higher than PHP.
What exactly would you expect from a table type that’s better than the “for row in fetchall()” style functionality? The database engines I know about will literally return a list of tuples to the client. Sqlite seems to allow stepping through this list to return the tuple at the current point a la “cursors”. Or is it more the row-number/column-name type access you’re aiming at here? That’s pretty trivial to implement with “magic methods”. For some weird reason that’s not the default behaviour in psycopg2 but you can ask for a dictionary-like cursor when making the db connection.
Ah, I see what you mean now. The distinction is about continuable versus non-continuable conditions/exceptions. That reminds me, I’d like to make continuable exceptions easier to handle in CHICKEN, as currently they’re a bit awkward to deal with right now.
Thankfully, most of us do :)
Think about what you’re doing inside those for-loops and how similar many patterns you might have there. Some of the ones I see every day:
Reorder (or build a permutation array) the table by one, two, many compound columns. In k, all vector operators are extended to tables by having tables implemented as a list of columns; arrays all the same length.
Join/zip two tables together (exceptionally common). Gross in Python, but in k this is just
,Determine if some table is inside another table.
Group some table rows by some columns.
Use a column as a rule on other columns. For example, if I want a table of switches, and one column to specify which switch to flip, in q this is
In Python, you need, if row-ordered something like:
or if column-ordered:
Neither is great, and I’ve seen worse contortions trying to rewrite it in SQL…
Deleting rows from a table (usually simulated with list.filter, but this conses a lot)
Upserting (often simulated with an index)
And so on.
One way to think about this: The “database engine” is just another programming language, and it’s another one that’s better than Python at tables. The challenge is to have tables in not a database engine.
It’s funny, because as soon as you put tables in there, plus “a little bit more”, you have a “database engine”, but exactly how much “a little bit more” is, and what you get for it makes you wonder why you just didn’t do it in Python (or C or Rust or Go whatever) to begin with.
Back in the day, the reason “the database” was separate was because a separate team managed it, but I feel like we’re past this now. Keeping tables out of languages isn’t some layer-violation, it’s a space of real problems people are working on every day that language designers are just ignoring for inertia reasons.
Well it’s one solution. I’m not closed to others, but it’s an obvious one that’s “better” by some definition. It’s worse by another (longer code) which might also be important: I put a fair amount of effort into fixing ECL’s file/IO “exceptions” to be restartable, but I don’t think anyone ever used them but me. Lisp is great, but people keep writing Java in it…
I’m still thinking about how language can help with this.
Hm, that’s thought-provoking, thank you for that! I see what you mean now, never really considered it to be a huge problem, but these patterns do occur over and over.
In most cases I would argue you didn’t write the right query to return just the data you need in the right form. Nine times out of ten you see people writing horribly contorted code that’s dog slow in their programming language (often performing tons of extremely dumb queries) to massage data into shape, in a way that could be done much better and more efficiently with SQL (often with a single query instead of many). So even if you have the right tools people will abuse them.
I would tend to disagree: the most important part (for me) about a database are the ACID properties offered by RDBMSes. It’s a separation of concerns; you don’t want your language to implement a storage engine, just like you don’t want your language to implement a raw file system; leave that to the OS kernel. You could do both in a library of course, but I’d much prefer a specialized separate tool to do it. Nevertheless, having good direct support for “tables” like you mention would be nice and a good and natural extension for dealing with result sets or tabular data that doesn’t need to go to an external database.
My hat’s off to you! We need more people to put effort into making existing good tools even better.
Yes. That happens very often. But I want to stress that tables don’t just live in the database.
I have an ad server that can either retrieve ads from its local database, or from a remote server (using HTTP). The publisher (client) requests ads from me, and I’ll basically be an aggregator. If the publisher asks for 10 ads, and the advertiser gives me 3, I might ask for 7 from another advertiser and glue them together.
The ad response is a table - each row representing a different ad, and each column representing some attribute (like the title, description, click-URL). I’m doing asof join (basically a merge-sort) to glue these together since I know the client may only take the first few ads so everyone who does this puts the higher-value ads at the top.
I’m given search keywords, so I do a query on this table – it’s a virtual table really, since it spans multiple databases (local and remote) but you can see this is clearly a table lookup made difficult by committee.
Knowing that, look at the schema Yahoo uses (starts around page 87). Do you see that table? How about when Google does it? Do you see the table now?
As soon as I discovered logging, I realised I never needed ACID Again. I rarely use an ACID database these days, and my application design has become a lot simpler as a result.
I remember I noticed that many business applications want their database to remember all the changes. And who made them. So many CRUD applications will in addition to all the insert/delete/update operations, duplicate those into an “audit” table of some kind. There may be even some business processes for recovering/rollback/cherry-picking values from that table.
However another way you can do this is to imagine you have a single “changes” table. Inserts to it are cheap! it’s just file append! Then all of your queries go to materialised views (re)built from those changes. In a catastrophic situation it’s trivial to replay the log and get the views back. Or change the schema and replay so you get your data right. No migrations.
It’s like patch theory for databases.
Here you can see we have atomicity when we want it (simply block queries to the new views while you’re building them), consistency (all the new views are built together), integrity (duh), and durability. In fact, this approach is so easy, you’re likely to get it right the first time. Putting b-trees on disks on the other hand is what all the “database engines” do, and to get acceptable performance means handling a lot of edge cases. That makes a lot of code- places for bugs to hide.
I’m not so sure. Being able to pickle/map disk-objects to values is incredibly valuable so why not a table?
k/q supports serialising all data types (including lambdas, btw) so they can be sent over the network or written to the disk. On the disk, a table can become a directory where each column is a separate file. mmap() brings them into memory and their layout is similar enough to their purely-in-memory version that few operations need to know the difference. This feels like an extreme version of what I used to get out of hibernation/pickling, and yet it’s so simple! And look at all I get!
Agreed, but I’d say the main downside (beside not amazing performance, even with PyPi) is the lack of meta-programming.
There’s some support for metaprogramming in the form of metaclasses, and you can do lots of magic with magic methods, iterators and context managers. There are even hooks that you can call when a class instantiates from an abstract base class. It simply isn’t part of the culture to do lots of metaprogramming, as magic is frowned upon as not being the “one obvious way to do it”. But there are projects that rely heavily on this stuff. For example, the lxml builder uses magic methods a lot, and Django uses model introspection to autogenerate database migrations.
I respect the attitude of Python culture of insisting on simple code (and to be fair, Python code tends to be very readable by default), but it’s not my favourite language. Python is relentlessly imperative and the functional style is really frowned upon, it has no macros (the existing metaprogramming is too clumsy to express certain things) and whitespace indentation makes working with the REPL a total pain in the neck.
I completely forgot about metaclasses, thanks for mentioning it. I actually learnt the basics of FP with Python 1.5 and the lambda/map/filter/reduce which at the time was already “ok, but don’t use it”. I continued to use it sporadically but found that list comprehension actually solved a lot of the use cases where I was using filter/map/reduce (although reduce in list comprehension is a bit of a headache). It’s weird, because I wouldn’t say Python is that imperative from a feature standpoint given that you can pass around functions, methods and classes (and now, even type descriptions), but seems to be, as you pointed it out, more culturally imperative. I wonder why that is, do you know happen to know the reasons behind this attitude/perception?
It seems Guido believes functional programming is hard to understand. Furthermore, it doesn’t fit his idea of Python, see this old Slashdot post (search for “functional programming”).
The need of a fast, static typed Python with metaprogramming is what drove me to Nim.
That was a surprisingly fun quick read.
As an example of another language that would foot the bill but be more…modern than the one described in TFA (no spoilers) would be REBOL.
It was the path not taken, sadly.
You might be aware, but Red is following that path. But they’ve gone off on a cryptocurrency tangent; I’m not quite sure what’s going on there anymore.
I think dialecting ala Rebol is super interesting, but I also think this sort of “wordy” input like AppleScript and DCL will eventually just become short forms that often require just as much effort to read later… that’s how you’d have things like
show device...foreshortened tosho dev ....Having said that, SRFI-10 or
#.form from CommonLisp is a happy medium, I think.I have not been responsible for a Cisco router in at least 15 years but I still find myself typing “sh ip int br” on occasion.
hahahaha oh lord, I know what you mean. I still have ancient devices burned in my brain as well, like OpenVMS and what not. Still, I think it goes to show that making things more “natural language-like” doesn’t really mean we want to write like that… there’s probably some balance to be struck between succinctness and power that we haven’t figured out yet
I also loved the bit of engagement at the end with the buttons. Been a string of really well written (light) technical articles lately, hope the trend continues.
I ported a REBOL app (using the full paid stack) to C# – the code inflation and challenge of making a 1:1 exact copy (no retraining port) was phenomenal. Most stuff took nearly an order of magnitude more code. There were some wins (dynamic layouts, resizing, performance) – but REBOL had shockingly good bang for the buck and dialects only really took a few days to grok.
I think Forth is really the language the 21st century needed. I see Forth as having little-to-no syntax, which gives it a wealth of expressiveness. Every Forth program is practically a DSL, or it isn’t, if you don’t want to write it like that. It’s typeless (though some implementations provide typing). Postfix is powerful. Any Forth library can expose a simple wordlist intended as the user API, and thanks to the hyperstatic environment avoid name conflicts with user code. Any data structures you like. It’s procedural, functional, stateful, stateless, dataflow-y, logical - whatever you want. Access to the return stack is equally as useful; mega fast (just skip all the cruft),
RDROP RECURSE, keeping arbitrary data out of the way for efficient stack usage. Some forths implement scoped variables if you want them (I know gforth does). I just think the main problem is the bickering community that doesn’t get much done, and the millions of implementations but lack of actual programs. The history of Forth is one of not adapting and not evolving. It’s more of a guideline, less a specific language, and that’s key to what Chuck Moore espoused it to be in his bookeven if the book is practically a tutorial on it’s implementationProgramming a Problem Oriented Language (https://colorforth.github.io/POL.htm). Maybe it’s too esoteric. I bet it would’ve blown up if someone had implemented tk bindings. Maybe someone should release a pre-installed Forth arduino. I really lost track of what I was going to say here, but I think some kind of point can be derived. Use Forth? Try it out at least.Or use SNOBOL. Or Java. Whatever.
I was expecting FORTH in that list too.
The author’s first and third points kind of contradict each other.
The whole point of creating a DSL is to make a language that mimics the end user’s domain. It’s analogous to how the English sentence syntax he recommends mimics a regular language. If it takes a lot of mental effort to learn a new DSL then it’s a bad DSL.
I also disagree with shrugging off performance concerns. Look at GMail’s new UI rollout - even in a high level web app, users still care about performance.
Sure, but GMail’s UI isn’t slow because it’s written in JavaScript. It’s slow because Google doesn’t give a fuck about making it fast.
COBOL has different semantics for
ADD,ADD TO,ADD TO GIVING, andCOMPUTE, as well asDIVIDE,DIVIDE INTO, andDIVIDE BY. I don’t think COBOL is the technical strawman he’s looking for, which undermines the (arguably valid) social point he’s trying to make.Yup. He’s made the same mistake that informed a lot of ‘english-like’ languages & confused ‘simple’ with ‘already familiar’, ignoring what needs to be unlearned to make coding possible. (How much easier would learning SQL be if the constructs didn’t resemble common english phrases & we therefore didn’t immediately expect to be able to reorder them?)
Entertaining read.
I’m thinking of ruby when reading this. If you remember _why’s guides, he was very enthusiastic about the natural language part of Ruby, and for me that was one of the things I liked about it. However, where it was easy to create API’s that felt like natural language, it also gave power to some syntactic nightmares. If you don’t believe me, just take a look at all Ruby submissions to the Advent of Code challenges, and you’ll find neat little one-liners that are impossible to parse by head.
I don’t know if the author is being sarcastic, but it fits COBOL to a T including the fixed point calculations, and it is even listed. Can OP confirm?
The buttons at the bottom are clickable.
The post is an extreme case of burying the lede.
Ah, missed that :(
I think the biggest issue these days is language longevity.
Creating a language and its surrounding ecosystem takes decades of work – and it’s usually ruined by people who just want to add yet-another language feature, until the language collapses under its own weight. Then the cycle repeats, with decades of effort wasted.
See Rust, C#, Scala, C++ … for the various stages of this.
There really needs to be some soul-searching/research on preventing this from happening over and over.
I disagree.
Vehemently.
“Readability” means I can read it.
An expert in those languages.
Not someone who doesn’t know those languages.
Readability is not code for “I can decode it with stackoverflow or google”
It doesn’t mean “it looks like every other language” either; That’s Approachability, and it may imply a level of Readability, it isn’t the same thing either.
We programmers can change ourselves to learn to Read new languages, and with practice and experience, can be many many times more successful* with “unreadable” languages than the Readable ones.
* That’s success as measured by personal metrics (salary, job satisfaction) and business metrics (time-to-deliver, cost of sales, risk), and not e.g. popularity.
Such an anticlimactic post.
The structured programming debate when they abolished goto. It was a pretty good improvement to the languages overall. It was only spoiled by the fact that “practical” people didn’t understand shit. The point was to make the code easier to understand by giving it regular structure. The complex structures could have been conveyed by tail recursion.
Well people found optimal tail recursion hard to implement, so instead they introduced ways to violate the rules that made structured programming work in the first place, the idea that you cannot haphazardly jump around in the code was weakened. They introduced break/continue into loops, and of course, the return statement. Well of course the structured programming was not as useful afterwards.
Likewise the object oriented programming was about processes and messages. Very important ideas to control for concurrent programming. People diluted it into a dogma that dominated the industry for few years until few figured they had chased red herrings there.
Of course if you fuck up with the theory part of your work, you’re certified to come up with tools that aren’t better than the things they replace. Jonathan Blow didn’t get this, Go completely missed it, Rust strike force almost figured it out. Whole C++ committee is out on the field and didn’t get the memo. Surprisingly it’s hard to grok that the theory is important.
If you’re going to try this the same way that you try to reinvent programming languages, the outcome won’t be different. For example of this you can just look at Bret Victor’s stuff or the EVE text editor. Also you’re going need a better language to come up with better tools. The language is your primary tool for being attentive on the detail and explaining complexity.
Sounds like a good reason to bang on schlub again.
Sounds like a call to use Mathematica.
Languages are tools. When I want to use a knife, I prefer a sharper, more durable, and more comfortable to use knife. Especially if I’m to use it ~8h each workday. Sure, I can probably do similar things with a blunt knife, but a better tool allows me to do more and faster. Unless I misunderstood something about the article, that would seem to be my answer to it - I don’t seem to see the point in the complaints. I know my knifes and there are a lot of interesting new ideas how better ones could be made. Rust, Luna (!), Elm, Zig, there are a lot of interesting new experiments that may (or already do) bring new quality to the table. There are also old knifes which sometimes get refurbished and strenghtened, improved and reinvigorated, and slowly transformed into something completely new, like OCaml/Reason, Red, Clojure, just from the top of my head. As to the general idea of “we should do better” - sure, but isn’t all life more or less about it? And better tools can help with this in various ways.