I’m the author of Clay.
@DRMacIver alerted me about this thread, and I thought I should post an update on the status of the project.
It’s kind of funny to see it being referred to as “Clay2011”, as 2011 was the year I actually stopped working on the project! (It was started a few years earlier).
Luckily, I’ve been able to allocate quality time to language development again in the last year. A new version of Clay is under very active development with a much better and simpler design. The compiler has been bootstrapped as of about a couple of months back. We’ve also started using Clay for our internal projects at Clay Labs.
Some notable improvements over the previous Clay are: Optional compiler enforced memory safety, and much faster compile times.
The compiler can parse all of itself, do whole program type-inference/checking, type specialization, and IR generation in a total of less than 2 seconds.
The backend code generation is still slow though. Producing an unoptimized binary of the compiler takes an additional 6 seconds. Producing an optimized binary takes quite a bit longer.
I would like to release this new version of Clay at some point in time when the advantages offered are meaningful enough for people to use it for real jobs. I’m not sure when that will be.
“I’m the author of Clay.”
I post some obscure, interesting work then one of the inventors shows up. What a treat!
“It’s kind of funny to see it being referred to as “Clay2011”, as 2011 was the year I actually stopped working on the project! ”
The year came from other Clay site saying it wasn’t the 2011 one. Ok, now it’s ClayFromSomeTimeBefore2011. Or maybe I’ll call it Sreeram’s Clay. That’s fitting. :)
“Luckily, I’ve been able to allocate quality time”
See DRMacIver: I knew they had the resources to be developing it and prototyping internal apps. Glad to see it’s happening. Modern, system languages just don’t get enough attention these days versus Yet Another Scripting/Managed Language.
“Optional compiler enforced memory safety” Ok, good.
“The compiler can parse all of itself… 2 seconds.” Great work.
“The backend code generation is still slow though”
Is your backend custom or are you leveraging an already-optimized compiler for this? Sounds like the easy route would be the latter. Especially on a Wirth-style compiler given they’re so fast. Another route is to let the compiler be a bit slow but include a REPL/interpreter for development iterations. There are even C interpreters for that. Once it’s solid, they run the optimizing compiler. It also usually has a feature catching the intermediate stuff and only compiling what’s necessary during a recompile.
Is your backend custom or are you leveraging an already-optimized compiler for this? Sounds like the easy route would be the latter. Especially on a Wirth-style compiler given they’re so fast.
Yep. I’m using Clang as the backend. The older Clay used LLVM but I switched to Clang so I don’t have to worry about the C ABI when interoperating with existing libs.
Another route is to let the compiler be a bit slow but include a REPL/interpreter for development iterations. There are even C interpreters for that. Once it’s solid, they run the optimizing compiler. It also usually has a feature catching the intermediate stuff and only compiling what’s necessary during a recompile.
I’m hoping to make the compilation itself faster. Since the front-end takes so little time, I can perhaps do some of the optimizations before delegating to the backend.
Sounds good. Btw, look up QBE compiler backend. It’s designed to be simpler IR than LLVM, totals around 10Kloc, and is selective about optimizations it includes to get 70% there in performance without much complexity. Might be nice interim solution.
QBE looks great. Thanks for the link.
I was disappointed to see the 2011 project is now dead after reading the authors' descriptions in the reddit ama.
It isn’t entirely clear to me if the ‘generic programming’ and ‘type propagation’ of Clay2011 is spiritually different from, say, the type inference system in Haskell. Is one more expressive? Is one more concise? Any insight? Different implementations with an equivalent result?
If the two are different, are there other languages with an approach like Clay2011?
Yeah, I used it a bit. It was good. It ran into a bunch of problems - AIUI primarily lack of funding/time to develop it. The primary author has made some promising noises about wanting to resume the project and looking for funding to do so, but I don’t know what the status of that is.
It isn’t entirely clear to me if the ‘generic programming’ and ‘type propagation’ of Clay2011 is spiritually different from, say, the type inference system in Haskell. Is one more expressive? Is one more concise? Any insight?
Clay’s generic programming can be better thought of as a rationalisation and improvement on C++ style compile time metaprogramming, only not terrible like that makes it sound. It’s much more ad hoc than the Haskell system, but also much more flexible. It’s quite pleasant to use, although does suffer a bit from the problem of it not always being very obvious what went wrong when it doesn’t work IIRC.
I’m not sure why they need funding given they spun it off into a business with interesting projects:
Maybe just not enough revenue coming in for sparing development time? One thing I suggest for situations like these is making, marketing, and licensing at least one app whose profits cover the cost of at least one developer (i.e. the inventor) to continue working on Clay with majority of time. More as sales increase. Ideally, the app could be written in Clay taking advantage of its features. Then, it and other projects later become success stories that promote the language. If uptake starts, then they can sell support for it directly.
I haven’t done an analysis to prove this model or anything. It just combines two, proven ones in a way that should imply the combo would work to sponsor a given project. In this case, developing the language. I’ll also add they should use one of the compiler or DSL frameworks that make this stuff easy to avoid compiler development being a time-sink of drudgery. I think that as default would help many new languages develop at least in prototyping phase where their features are in flux and they have stuff to prove.
I’m not sure why they need funding given they spun it off into a business with interesting projects:
AIUI it’s the other way around. The language came out of the labs, and most of their projects are not using it. I don’t really have any inside knowledge on the situation though so may have the details entirely wrong.
Yeah, they built it. I’m not implying they’re doing the stuff in Clay. I’m just saying the Clay team has a business bringing in income on high-performance, server apps. Both could benefit it or from it in some way. They must just not have a lot of money coming in. Likely, they’re charging for their time instead of I.P.. Often does it.
The author gave a great, brief description of it here:
That and sub-thread with Felicia may give you what you need to do the comparison.
just like every other ‘for future programming’ and ‘innovative’ new programming languages nowadays
Joe Groff, the owner of the 2011 repository, now works for Apple on Swift.
Ahhhhhh. That makes a lot of sense if wondering why his development would halt or slow. I’d definitely choose to work on Swift over Clay2011 if I had a choice given money and momentum behind Swift. More impact, too. Explains him but not Clay labs. I might just email them.
Meta note for moderators & regulars: appreciate feedback in this comment thread on the double, submission concept. I figured them showing up separately might cause confusion. Might be fine where someone just adds in comments “not to be confused with…” Alternatively, as I did here, just bring both to people’s attention simultaneously with relevant links clearly marked to avoid confusion. The naming should avoid it in comments but who knows.
Another thing to do would just be to submit the story here directly (without a URL) and put the writeup in the description–that way, no pastebin needed.
Never would’ve occurred to me! So, can you do hyperlinks and all that in “description” on submit page?
Yes, and you can even tell from the lobste.rs homepage which threads include a description – they have a ☶ character next to them.
Yes, it’s the same Markdown variety as ordinary comments. There’s a link below the box that pops up the formatting syntax.
I don’t think it’s necessary to give a lot of feedback because it’s hard to imagine there will be many cases where you want to submit stories about two programming languages that have the same name. Not that the phenomenon is rare, just the need to disambiguate. :)
That makes sense. So do you think this kind of double submit is a good way to handle those exceptions? And your thoughts on alternatives like angersocks'?
The description field is useful and I wish people would use it more. :) I especially love when it’s used for a brief explanation of why this audience might be interested in the piece. I would prefer that over the pastebin, which was an extra click and not really formatted for easy reading, but I’ve certainly used links to pastebins elsewhere and they have their uses.
But this is just one community member’s opinion; I think watching what gets upvotes is the best way to get a sense of what works well for people.
On mobile, pastebin doesn’t already link, so it’s far more than one extra click. :+1:
Never occurred me. Thanks for telling me. That psychological element might have factored into some of my Pastebin’s on Hacker News where they seemed to not have read the citations. On top of normal reasons people don’t fact check. ;)
What Irene said, and also that it can be very useful to add background links to wikipedia or previous discussions as well.
I forgot to tell you about the description field. I think it’s due to how the site presents it that some, including me, avoid it. I’d prefer it used as you mentioned as many love a brief summary to prevent wasting time. When I submitted it, the box said the following:
“optional when submitting a URL; please see the guidelines”
Also a link to show the guidelines. First says “don’t editorialize story titles.” May refer to the title or description boxes. First clearly about text field implies it should only be used if absolutely necessary with important information in comments. So, I thought we weren’t allowed to do what you suggest per guidelines. So, I created a Pastebin about two languages then made a title that matched it without need to editorialize or elaborate any further. I put the further elaboration into the Pastebin, but usually I do in a comment per guideline.
Maybe new wording on the guidelines is in order.