I’m very super extremely interested in Garnet since I first learnt about it from your side-note at some point in the past (possibly in “What are you doing this weekend”, or some other discussion). I’m a big fan of Rust & Lua (recently employed in the former), and also Go and Nim before (and still to some extent), but totally not a CompSci person :) With this context, does the “I guess it’s time to start advertising” note maybe mean it’s at a stage where I could try experimenting with it or something? This is one thing that is quite unclear to me from the linked page, and I’d expect to be more so (i.e. what state is it at). Also: please note the ritual question/whine of: Why Doesth Ye Landingth Pageth Noth Showeth A Sampleth Codeth Snippeth™?
With this context, does the “I guess it’s time to start advertising” note maybe mean it’s at a stage where I could try experimenting with it or something?
Sadly… not really. It’s more that I’m sick of working in a vacuum, and I’m sick of people writing blog posts like the “Thoughts on Zig” one where I read them and go “I’m trying, dammit!”. It’s in the “technically Turing complete but doesn’t really do much” phase, I’m just vaguely optimistic that the ideas are actually fairly solid now, and it will progress to Interesting Things pretty rapidly once I nail down type checking.
Also: please note the ritual question/whine of: Why Doesth Ye Landingth Pageth Noth Showeth A Sampleth Codeth Snippeth™?
I’ve been watching this page and your posts in the r/PL discord, and I’m excited for the direction this is going in! I’ve got some notes / questions on what you have so far.
Better ergonomics around borrowing would be nice too, though I’m not sure how to do that yet, I just hate that there’s no way to abstract over ownership and so we have Fn and FnMut and FnOnce.
[…]
You can’t be generic over mutability and ownership, so for example you end up with iter(), into_iter(), and iter_mut().
What would a function that is generic over mutability and/or ownership look like? What could you do with a param that you don’t know if you have mutable access to and don’t know if you own? Isn’t the explicitness the whole point of having these be separate pointer types? If Garnet had some way to reify the information then I think this could be useful, but I’d have to see some example implementations. Personally, I think the iter() trio has some bad names but the semantic difference is important, and I think it would be lost if that function was generic.
Additionally, will Garnet implement lifetime tracking? One idea I can’t grok is how you can minimize Rust’s complexity without losing required language features. For example, to any C / Lua programmer, Higher-Rank Trait Bounds must seem like an absolute nightmare. Higher-Rank anything is dipping your toes into some meaty theory that “practical programmers” dislike. But without HRTB, there would be some functions with lifetimes that your language would be incapable of expressing. It seems to me that much of the complexity of Rust stems the intersection of traits and lifetimes.
Again, this is kinda similar to Zig, but I want to avoid the problem where you don’t know whether the code will work before it’s actually instantiated.
Zig does this for conditional compilation / comptime tree shaking. Since you’re going to need to compile some code that uses Linux’s ABI and some that uses Windows’ ABI, I think some conditional compilation is going to be required for a systems language. Rust uses build-time configuration flags (#[cfg(...)]) which, imo, is a hack. I think Zig’s approach is a lot simpler, even if it requires lazy resolution.
Another thing to note here is testing - idiomatic Zig uses test {} blocks to ensure your code is actually being resolved.
fn Vec(comptime len: comptime_int) type {
return struct {
const LEN = len;
// ...
};
}
test "resolve Vec" {
_ = Vec(5);
}
Tests can also be skipped on certain OSes (w/ something like return something.SkipTest?). This has the same functionality as Rust’s cfg, but doesn’t require obscure annotations on your functions. I prefer this method because it makes the language smaller.
Rust’s trait orphan rules are annoying, but may be too hard to be worth trying to solve.
Trying out QBE and Cranelift both seem reasonable choices, and writing a not-super-sophisticated backend that outputs many targets seems semi-reasonable.
I wrote a toy language with an IR that translated to LLVM and CraneLift and it wasn’t too bad. CL’s API was pretty nice. One thing to note is CL uses extended basic blocks instead of phi nodes. I found that having my IR use EBB and compiling that to phi nodes for the LLVM backend was easier than turning basic blocks w/ phi nodes into EBB.
I’ve been watching this page and your posts in the r/PL discord
oh no
What would a function that is generic over mutability and/or ownership look like?
I don’t know yet, to be quite honest. I don’t think it would technically add any power to the language. But if you have something like, in Rust, fn iter(&self) -> impl Iterator<Item=&T> and fn iter_mut(&mut self) -> impl Iterator<Item=&mut T> and fn into_iter(self) -> impl Iterator<Item=T> then having three different methods really doesn’t get you much in terms of power either. The important information is that the ownership of the output T is the same as the ownership of the input self. What can you do with the param? Mostly just move and assign it, but sometimes that’s enough for the call-ee. The caller knows the ownership status of the self in question and so knows what it’s getting.
As for what it will look like and how it will work, I’m open to suggestions. The above statement is literally about as far as I’ve gotten on it.
Additionally, will Garnet implement lifetime tracking? …
It will definitely track lifetimes, and I don’t actually think (anymore) I’ll be able to improve on Rust’s lifetimes that much, but it’s been a while since I visited this problem in depth. I agree that lifetimes + traits = complexity, but more of my work has been spent gnawing on the trait side of that equation than the lifetime side. Higher ranked trait bounds are a good example though, since I’ve been using Rust since the 1.0 release and I’ve had to write code that uses HRTB’s about twice, and both times I ended up refactoring my code so that I didn’t need it because it was such a pain in the ass to actually figure out. So is the cost of writing unsafe { ... instead of for<'a> ... really that big? Opinions will inevitably vary, but for the moment I’m leaning towards “no HRTB’s if possible”.
… I think some conditional compilation is going to be required for a systems language. …
It is, but I haven’t really figured out what form it would take. My thought for lack of anything better was “steal Rust’s system”, which usually gets you at least a half-decent default. I hadn’t considered using lazy type instantiation for the same sort of thing, it’s an interesting approach. Rust’s #[cfg(...)] blocks certainly have all the downsides of “well this works on my platform” that scares me about Zig’s type resolution, so… idk, might be worth looking into more.
…idiomatic Zig uses test {} blocks to ensure your code is actually being resolved.
…
…but doesn’t require obscure annotations on your functions.
From the outside I don’t see much to choose one over the other.
Rust’s trait orphan rules are annoying, but may be too hard to be worth trying to solve.
I just wanted to note that Carbon decided to do the same thing here.
Actually, with ML modules, orphan rules go away because choosing implementations of your trait-ish things is explicit. The classic example of trait conflicts that I like is you have module A that provides type A.T and implements Hash on it, then you have module B that implements its own copy of the Hash trait on type A.T with a different hashing algorithm. Now whenever you get an A.T and pass it to something you don’t know whether it should use A’s implementation of Hash or B’s implementation of Hash
This doesn’t happen with modules ‘cause it extracts the implementation from the type. Rust-style traits basically have a list of implementations of traits attached to each type, which is global. ML modules have a list of implementations of traits that are not attached to particular types, they live in your module namespace like any other value, so if your function is passed an A.T and wants to hash it then it also has to know what implementation its using. If something creates an Hashset<A.T, A.Hash> or whatever and passes it to a function that is using B.Hash then it won’t typecheck.
The downside of this is that juggling this by hand instead of using the type as your namespace is kinda a pain in the ass. I have yet to make it particularly satisfactory. So, we will see. My most recent “this is what I want to be able to make compile” experiment is here, I think.
I wrote a toy language with an IR that translated to LLVM and CraneLift and it wasn’t too bad.
Good to know! Cranelift was a fair bit less mature when I started this project but I continue watching it with interest from time to time. The only downside right now, besides stability (which will be true for any compiler backend I expect), is that it’s intending to optimize for JIT use cases rather than AOT compilation. But maybe I’ll give it a go. I do slightly prefer EBB’s to phi nodes, though for now my attempts at writing an optimizing backend basically made me go “woah I need to learn more about control-flow and data-flow analysis” and back off a bit.
My thought for lack of anything better was “steal Rust’s system”, which usually gets you at least a half-decent default.
For a while now, I’ve been thinking that it would be nice to have a wiki that lists and compares implementations of programming language features for programming language designers. Anybody know if that already exists?
This is a great goal and I’m excited you’re working on it!
Better ergonomics around borrowing would be nice too, … However, trading some runtime refcounting/etc for better borrowing ergonomics as suggested in Notes on a Smaller Rust and Swift’s work is not on the cards
…but, that’s a bit disappointing. My takeaway from spending a couple months each with Rust and Nim was that strict borrow-checking adds too much complexity to the language and frustration to the programmer’s job. I would much rather let some things have hidden refcounts than spend my day arguing with the compiler about how to optimize everything’s lifespan. (Yes I do care about performance, working on stuff like database engines, but not to that degree.)
I also don’t see how you can go simpler and still have some degree of memory safety, without at least lightweight (refcounted) GC, but I’m also not a PLT expert…
…However, trading some runtime refcounting/etc for better borrowing ergonomics as suggested in Notes on a Smaller Rust and Swift’s work is not on the cards…
…but, that’s a bit disappointing.
Sorry. Garnet isn’t for your use case then. 😅 Maybe the next language I make…
I also don’t see how you can go simpler and still have some degree of memory safety, without at least lightweight (refcounted) GC…
I’m not sure you can, at least by much! I was more sure when I first started the language… But also see my other post talking about higher-ranked trait bounds. I think there’s also room to explore. What would a Really Immutable language a la Erlang or Haskell look like, with a borrow checker? How much would it need to actually use its GC, vs what its borrow checker could figure out at compile time? I dunno!
Well, Garnet certainly ticks a lot of boxes for me: compile-time memory safety, enums, no nill / null, relatively small (the language, the implementation). You mentioned Lua in the README, so that is a bonus point in my book.
This is really interesting to me.. I like Rust a lot, but I also find I miss the ability I had in C, to fairly reliably predict what assembly would be generated from the input source. A good thing about Rust is that its design process is very open, so when you have things like:
I just hate that there’s no way to abstract over ownership and so we have Fn and FnMut and FnOnce.
For my part, I’ve wanted a Rust with linear types since forever, and started work on it in private, but I’ve struggled with making it sound (in the presence of safe forget) and ergonomic (in the presence of non-abort’ing panic).
Something that Zig has that Rust doesn’t is “required” error handling. I know you can mark functions with error reports as “must_use” or whatever, but with Zig it’s not up to the programmer to remember to do so. It’d be neat if that were in Garnet. :)
I’ll have to look at it more. My initial impression of Zig’s error handling was vague distaste that it was essentially a side-channel to normal code. Never used it in anger though, so that might be something I have to do soon.
Oh, that’s ‘cause it’s not terribly interesting in terms of the language. …and because the stdlib doesn’t actually exist yet. You can use something like this:
fn main(): {} =
__print_str("hello world")
end
The __print_str() function is a compiler built-in for the moment.
If by “type inference” you mean real whole-program (instead of the weaker local stuff found in eg rust) and you keep the full strength borrow checking, this may be the first real entrant to the rust part of the design space. Very interested
I agree; it’s a small thing but pervasive, and one of the first things anyone will notice about the language. I suppose Rubyists will feel at home, but no one else.
IMO the reasonable choices would be either curly-braces like nearly every mainstream language, or indentation-based like Python and Nim. But I’m not the one writing the parser :)
If you make sure your code can be evaluated at compile time, you can treat types mostly like normal values, and “instantiating a generic” becomes literally just a function that returns a function or struct.
I dislike this. I think metaprogramming is an initially attractive idea, but not worth the complexity that it encourages in the entire ecosystem. I am extremely happy with the results of omitting it from Myrddin.
I’d argue this approach is simpler than both traditional generics and not having generics at all. Why would functions be arbitrisarily limited to having non-types as arguments?
Because it simplifies the code that gets written by taking away room for excessive cleverness, which makes it easier to read and understand what is going on.
This argument could be used against literally any feature. Better not have functions at all, or someone might use them for “excessive cleverness”.
Yes. And your counterargument could be used to argue for allowing any feature in.
And this is where engineering happens: Deciding which features pay for their costs. Experience shows me that metaprogramming is not one of those features.
I am for including features as long as they’re useful and can’t be implemented using something more general. Metaprogramming is actually great because it can replace many other features, which often causes a net reduction in complexity. The problems you mentioned will only appear if you’re an irresponsible programmer who doesn’t follow the principle of least power.
I have found that metaprogramming adds many nasty edge cases to implementations, and is used well extremely rarely. Most features you can implement with both compiler features and intrinsics are best omitted entirely.
I largely don’t want these features that metaprogramming generalizes. Not in the compiler, and not as a library. The one that I do is type constraints – which is very difficult to implement as a library, because it implies attaching extra data to a type when type checking – but it’s simple to implement within the compiler with a bit and, once the extra data is part of the type. (Simple, as in it adds about 20 lines to the type checker, last I remember).
If, as you say, the principle of least privilege means that they will be nearly always used sparingly in boring ways, then they are complex dead weight in the compiler. If that’s not how they’re used, then they are a weight dragging down the code.
Metaprogramming is definitely a spicy feature. Attempts at excessive metaprogrammingin Rust libs has certainly driven me up the wall in the past. But at a glance Myrddin has traits, from what I see as well, which is already metaprogramming? I certainly sympathize with wanting to keep life simple.
I make no promises that this is the best approach, just a reductionist one that seems to work okay thus far.
Code generation is only as messy as your target language. The complexities come from alpha-equivalence, substitution, typing, quotation, and linking. If you can manage to avoid those issues by designing a simpler target language, then code generation is not a big deal. This is especially true for languages with an eval() tool.
People keep writing stuff like the recently-posted https://v5.chriskrycho.com/journal/some-thoughts-on-zig/, so I figured it’s time to start talking about this for real.
Edit: Added some more info to readme to answer common questions, commit 13f6c2f14ab1a4a6aac31f176b0d25b5b0227a2b
That Graydon Hoare quote at the end is really wholesome. Good luck on garnet!
Well you certainly got my attention with it. 😅
really wonderful to see this space being explored! garnet looks like it’s going to be great, looking forward to using it someday.
Instantly noted. Analysis soon.
oh shit. uh, be gentle?
I’m very super extremely interested in Garnet since I first learnt about it from your side-note at some point in the past (possibly in “What are you doing this weekend”, or some other discussion). I’m a big fan of Rust & Lua (recently employed in the former), and also Go and Nim before (and still to some extent), but totally not a CompSci person :) With this context, does the “I guess it’s time to start advertising” note maybe mean it’s at a stage where I could try experimenting with it or something? This is one thing that is quite unclear to me from the linked page, and I’d expect to be more so (i.e. what state is it at). Also: please note the ritual question/whine of: Why Doesth Ye Landingth Pageth Noth Showeth A Sampleth Codeth Snippeth™?
Sadly… not really. It’s more that I’m sick of working in a vacuum, and I’m sick of people writing blog posts like the “Thoughts on Zig” one where I read them and go “I’m trying, dammit!”. It’s in the “technically Turing complete but doesn’t really do much” phase, I’m just vaguely optimistic that the ideas are actually fairly solid now, and it will progress to Interesting Things pretty rapidly once I nail down type checking.
Oh yeah, that’s a good idea. There’s a bunch of small programs in its test suite here: https://hg.sr.ht/~icefox/garnet/browse/tests/programs?rev=12ee941c3da958f037ba0a9509d0ebc00c6c0465 And some slightly-more-interesting-but-sometimes-still-hypothetical programs here: https://hg.sr.ht/~icefox/garnet/browse/gt?rev=12ee941c3da958f037ba0a9509d0ebc00c6c0465
I’ve been watching this page and your posts in the r/PL discord, and I’m excited for the direction this is going in! I’ve got some notes / questions on what you have so far.
What would a function that is generic over mutability and/or ownership look like? What could you do with a param that you don’t know if you have mutable access to and don’t know if you own? Isn’t the explicitness the whole point of having these be separate pointer types? If Garnet had some way to reify the information then I think this could be useful, but I’d have to see some example implementations. Personally, I think the
iter()trio has some bad names but the semantic difference is important, and I think it would be lost if that function was generic.Additionally, will Garnet implement lifetime tracking? One idea I can’t grok is how you can minimize Rust’s complexity without losing required language features. For example, to any C / Lua programmer, Higher-Rank Trait Bounds must seem like an absolute nightmare. Higher-Rank anything is dipping your toes into some meaty theory that “practical programmers” dislike. But without HRTB, there would be some functions with lifetimes that your language would be incapable of expressing. It seems to me that much of the complexity of Rust stems the intersection of traits and lifetimes.
Zig does this for conditional compilation / comptime tree shaking. Since you’re going to need to compile some code that uses Linux’s ABI and some that uses Windows’ ABI, I think some conditional compilation is going to be required for a systems language. Rust uses build-time configuration flags (
#[cfg(...)]) which, imo, is a hack. I think Zig’s approach is a lot simpler, even if it requires lazy resolution.Another thing to note here is testing - idiomatic Zig uses
test {}blocks to ensure your code is actually being resolved.Tests can also be skipped on certain OSes (w/ something like
return something.SkipTest?). This has the same functionality as Rust’scfg, but doesn’t require obscure annotations on your functions. I prefer this method because it makes the language smaller.I just wanted to note that Carbon decided to do the same thing here.
I wrote a toy language with an IR that translated to LLVM and CraneLift and it wasn’t too bad. CL’s API was pretty nice. One thing to note is CL uses extended basic blocks instead of phi nodes. I found that having my IR use EBB and compiling that to phi nodes for the LLVM backend was easier than turning basic blocks w/ phi nodes into EBB.
oh no
I don’t know yet, to be quite honest. I don’t think it would technically add any power to the language. But if you have something like, in Rust,
fn iter(&self) -> impl Iterator<Item=&T>andfn iter_mut(&mut self) -> impl Iterator<Item=&mut T>andfn into_iter(self) -> impl Iterator<Item=T>then having three different methods really doesn’t get you much in terms of power either. The important information is that the ownership of the outputTis the same as the ownership of the inputself. What can you do with the param? Mostly just move and assign it, but sometimes that’s enough for the call-ee. The caller knows the ownership status of theselfin question and so knows what it’s getting.As for what it will look like and how it will work, I’m open to suggestions. The above statement is literally about as far as I’ve gotten on it.
It will definitely track lifetimes, and I don’t actually think (anymore) I’ll be able to improve on Rust’s lifetimes that much, but it’s been a while since I visited this problem in depth. I agree that lifetimes + traits = complexity, but more of my work has been spent gnawing on the trait side of that equation than the lifetime side. Higher ranked trait bounds are a good example though, since I’ve been using Rust since the 1.0 release and I’ve had to write code that uses HRTB’s about twice, and both times I ended up refactoring my code so that I didn’t need it because it was such a pain in the ass to actually figure out. So is the cost of writing
unsafe { ...instead offor<'a> ...really that big? Opinions will inevitably vary, but for the moment I’m leaning towards “no HRTB’s if possible”.It is, but I haven’t really figured out what form it would take. My thought for lack of anything better was “steal Rust’s system”, which usually gets you at least a half-decent default. I hadn’t considered using lazy type instantiation for the same sort of thing, it’s an interesting approach. Rust’s
#[cfg(...)]blocks certainly have all the downsides of “well this works on my platform” that scares me about Zig’s type resolution, so… idk, might be worth looking into more.From the outside I don’t see much to choose one over the other.
Actually, with ML modules, orphan rules go away because choosing implementations of your trait-ish things is explicit. The classic example of trait conflicts that I like is you have module
Athat provides typeA.Tand implementsHashon it, then you have moduleBthat implements its own copy of theHashtrait on typeA.Twith a different hashing algorithm. Now whenever you get anA.Tand pass it to something you don’t know whether it should useA’s implementation ofHashorB’s implementation ofHashThis doesn’t happen with modules ‘cause it extracts the implementation from the type. Rust-style traits basically have a list of implementations of traits attached to each type, which is global. ML modules have a list of implementations of traits that are not attached to particular types, they live in your module namespace like any other value, so if your function is passed an
A.Tand wants to hash it then it also has to know what implementation its using. If something creates anHashset<A.T, A.Hash>or whatever and passes it to a function that is usingB.Hashthen it won’t typecheck.The downside of this is that juggling this by hand instead of using the type as your namespace is kinda a pain in the ass. I have yet to make it particularly satisfactory. So, we will see. My most recent “this is what I want to be able to make compile” experiment is here, I think.
Good to know! Cranelift was a fair bit less mature when I started this project but I continue watching it with interest from time to time. The only downside right now, besides stability (which will be true for any compiler backend I expect), is that it’s intending to optimize for JIT use cases rather than AOT compilation. But maybe I’ll give it a go. I do slightly prefer EBB’s to phi nodes, though for now my attempts at writing an optimizing backend basically made me go “woah I need to learn more about control-flow and data-flow analysis” and back off a bit.
For a while now, I’ve been thinking that it would be nice to have a wiki that lists and compares implementations of programming language features for programming language designers. Anybody know if that already exists?
This is a great goal and I’m excited you’re working on it!
…but, that’s a bit disappointing. My takeaway from spending a couple months each with Rust and Nim was that strict borrow-checking adds too much complexity to the language and frustration to the programmer’s job. I would much rather let some things have hidden refcounts than spend my day arguing with the compiler about how to optimize everything’s lifespan. (Yes I do care about performance, working on stuff like database engines, but not to that degree.)
I also don’t see how you can go simpler and still have some degree of memory safety, without at least lightweight (refcounted) GC, but I’m also not a PLT expert…
Sorry. Garnet isn’t for your use case then. 😅 Maybe the next language I make…
I’m not sure you can, at least by much! I was more sure when I first started the language… But also see my other post talking about higher-ranked trait bounds. I think there’s also room to explore. What would a Really Immutable language a la Erlang or Haskell look like, with a borrow checker? How much would it need to actually use its GC, vs what its borrow checker could figure out at compile time? I dunno!
Well, Garnet certainly ticks a lot of boxes for me: compile-time memory safety, enums, no nill / null, relatively small (the language, the implementation). You mentioned Lua in the README, so that is a bonus point in my book.
I’ll be looking more closely at it.
Thanks for sharing.
This is really interesting to me.. I like Rust a lot, but I also find I miss the ability I had in C, to fairly reliably predict what assembly would be generated from the input source. A good thing about Rust is that its design process is very open, so when you have things like:
you can fairly easily find Rust community discussion about the issue, like the links from https://stackoverflow.com/a/41445002 .
For my part, I’ve wanted a Rust with linear types since forever, and started work on it in private, but I’ve struggled with making it sound (in the presence of safe
forget) and ergonomic (in the presence of non-abort’ingpanic).Garnet looks lovely.
Something that Zig has that Rust doesn’t is “required” error handling. I know you can mark functions with error reports as “must_use” or whatever, but with Zig it’s not up to the programmer to remember to do so. It’d be neat if that were in Garnet. :)
I’ll have to look at it more. My initial impression of Zig’s error handling was vague distaste that it was essentially a side-channel to normal code. Never used it in anger though, so that might be something I have to do soon.
How does this work in the face of generic code? And how exactly is ‘handling’ qualified?
What does the classic
Hello, World!program look like in Garnet?I’m trying to add it to my little editor named
o.I browsed through the examples but could not spot it.
Oh, that’s ‘cause it’s not terribly interesting in terms of the language. …and because the stdlib doesn’t actually exist yet. You can use something like this:
The
__print_str()function is a compiler built-in for the moment.Thanks! Added!
I wonder if
ois the first editor to specifically support Garnet.I will implement “jump to error” as well, if
garnetcsupports this.If by “type inference” you mean real whole-program (instead of the weaker local stuff found in eg rust) and you keep the full strength borrow checking, this may be the first real entrant to the rust part of the design space. Very interested
semi-pascal blocks are worse than either full pascal, curly braces, or semantic indentation
I agree; it’s a small thing but pervasive, and one of the first things anyone will notice about the language. I suppose Rubyists will feel at home, but no one else.
IMO the reasonable choices would be either curly-braces like nearly every mainstream language, or indentation-based like Python and Nim. But I’m not the one writing the parser :)
Feel free to open an issue or submit a patch.
Anything is better than semantic indentation. Even LISP is better :P
Anything is better than braces*
For those keeping score at home, everything is terrible and we are all doomed.
You mean having loads of ‘begin’ statements everywhere? Pascal’s successors don’t have that either. Only on the unit/proc level
Very cool, excited to see where this goes!
I dislike this. I think metaprogramming is an initially attractive idea, but not worth the complexity that it encourages in the entire ecosystem. I am extremely happy with the results of omitting it from Myrddin.
I’d argue this approach is simpler than both traditional generics and not having generics at all. Why would functions be arbitrisarily limited to having non-types as arguments?
Because it simplifies the code that gets written by taking away room for excessive cleverness, which makes it easier to read and understand what is going on.
This argument could be used against literally any feature. Better not have functions at all, or someone might use them for “excessive cleverness”.
Yes. And your counterargument could be used to argue for allowing any feature in.
And this is where engineering happens: Deciding which features pay for their costs. Experience shows me that metaprogramming is not one of those features.
I am for including features as long as they’re useful and can’t be implemented using something more general. Metaprogramming is actually great because it can replace many other features, which often causes a net reduction in complexity. The problems you mentioned will only appear if you’re an irresponsible programmer who doesn’t follow the principle of least power.
I have found that metaprogramming adds many nasty edge cases to implementations, and is used well extremely rarely. Most features you can implement with both compiler features and intrinsics are best omitted entirely.
I largely don’t want these features that metaprogramming generalizes. Not in the compiler, and not as a library. The one that I do is type constraints – which is very difficult to implement as a library, because it implies attaching extra data to a type when type checking – but it’s simple to implement within the compiler with a bit and, once the extra data is part of the type. (Simple, as in it adds about 20 lines to the type checker, last I remember).
If, as you say, the principle of least privilege means that they will be nearly always used sparingly in boring ways, then they are complex dead weight in the compiler. If that’s not how they’re used, then they are a weight dragging down the code.
Metaprogramming is definitely a spicy feature. Attempts at excessive metaprogramming in Rust libs has certainly driven me up the wall in the past. But at a glance Myrddin has traits, from what I see as well, which is already metaprogramming? I certainly sympathize with wanting to keep life simple.
I make no promises that this is the best approach, just a reductionist one that seems to work okay thus far.
Traits are nothing more than a filter on acceptance types.
“Any type such that type has this set of functions available”, and nothing more. It can be implemented a number of ways, but that’s all it allows.
In the end, it’s just a bitset with an AND for type checks. Code gen is more complex.
Code generation is only as messy as your target language. The complexities come from alpha-equivalence, substitution, typing, quotation, and linking. If you can manage to avoid those issues by designing a simpler target language, then code generation is not a big deal. This is especially true for languages with an
eval()tool.