as what seems to me a general way to implement add-on meta programming for languages which are not based around meta in the first place, which is simultaneously:
I wonder why they are calling them āmacrosā when they are more like python decorators or java annotations, much less like the syntactic replacement that āmacroā suggests.
I guess the actual reason is the same why Rust calls sum types enums ā itās more important to have succinct word which is clear in context, then to pedantically follow etymology.
Otherwise, I donāt think this is like āannotationsā, more like āannotation processorsā. And given that the primary reason for the processing would be to generate source code (so, not runtime functions, like in the case of pythonās decorators), macros seem defensible to me!
Is there a way to have type safe macros? I love gleamās error messages and precise inference. Iām afraid macros could lead to what I personally donāt like about elixirās usage of macros: debugging gets harderā¦
What do you think of adding module types and functions a la Ocaml? I donāt mean the full system, something in between erlang behaviors and a typed structure.
The full system doesnāt fit well with BEAM due to lack of full program compilation to dedupe generated code, and just typing modules is very limited due it being a weaker version of the record type.
Iām reacting to the headline which made me think there would be any metaprogramming content in the post. Iām pointing out to anyone else interested on that basis that there isnāt any. This is the old āif you want that try submitting an RFC/PEP.ā
@lpil If you implemented the BABLR vm you could gain the ability for gleam to metaprogram itself and also (potentially) every other programming language that exists at the same time
I wouldnāt be surprised if comments of this form were the motivation behind the āCompilers are just programsā section. š
Itās massively easier to say ājust implement this VM and you get all of these benefits!ā than to to change your widely-used compiler to actually do that, deal with all the technical and social ramifications of that change (what does using that VM do to compile times? To runtime performance of compiled programs?), and so on.
Sorry, in my excitement last night I donāt think I offered an adequate response to your technical query. The VM Iām talking about isnāt a runtime VM; instead itās an edit-time VM not unlike tree-sitter. I believe BABLR to be more generally applicable than tree-sitter though, in part due to the lack of error-recovery during parsing
Yeah I know that a one sentence comments probably seems a bit like sniping form the sidelines, but Iād guess that Iāve invested 15,000 hours so far. Four years full time without pay, costing me my lifeās savings. I just completed the sale of my airplane to have a few more months to work.
Now what seemed an impossibly high obstacle now seems more like what you describe ā a set of practical concerns related to how to deploy the tech
My expectation isnāt that parser authors will do the work to support my embedding format, but rather my embedding format will eventually provide so much value that people will fork parsers just to cause them to return data in a standard embedding format. I even expect Zed to adopt it eventually
What you are doing looks interesting but I must say that if you are aiming for wide adoption (spoken as a maintainer of a popular text editor) you need to use something else than Javascript.
LSP and TS are quite popular because itās easy to embed them into very different applications (TS because itās a C lib and can be linked to directly or compiled to WASM, LSP because itās just a JSON IPC API). Javascript is going to be a complete nonstarter for editors like (n)vim, emacs, helix,ā¦
You donāt need JS at all to be able to use CSTML as an embedding format for code. My goal right now is to provide a reference implementation. Once the reference implementation provides a firm target for interoperability I fully expect native implementations to crop up, perhaps as new projects or perhaps as adaptations to existing tools like Tree-sitter or Biome.
BABLR is a parser/grammar layer on top ā a generic system that allows grammars to be defined that emit CSTML parse results. Ideally its results will be immutable, which allows you to think of it as something not unlike a system of XML schema validators.
Hereās where it gets interesting. Once you have an embedded CSTML representation the BABLR VMās APIs are defined in terms of instructions expressed as CSTML, so any scripting language which has access to the embedded form of CSTML can be used to steer execution of a native BABLR core as it chews up text and emits parse trees.
I know this works because even in JS BABLR is implemented as a bootsrapping compiler
I think you have a lot of material and itās all a little unclear how it fits together. I think you need a worked example with detailed docs explaining how you added this on to something else and made it appreciably more awesome than it was before.
Oh absolutely! Iām nearly ready to set the standards in stone. A high degree of accessibility creates lock-in on standards, so Iāve been very cautious in that way thus far.
Just wanted to add that I read the BABLR README, and Iām a huge lang nerd, and I still couldnāt tell what Iām looking at. It sounds like an LSP alternative but beyond that not sure.
Thatās ok! If I can do anything to make it more clear I am at your service. The goal is to provide a narrow waist between tools and languages, and LSP is currently the closest we have. CSTML isnāt modeled on LSP at all though ā itās a new kind of thing which is technologically most closely related to XML and XML schema validators
Funny thing is the post is not specifically about metaprogramming but more of a meta guide for people who want to change things in the language š
Highly recommend
https://github.com/dart-lang/language/blob/main/working/macros/feature-specification.md
as what seems to me a general way to implement add-on meta programming for languages which are not based around meta in the first place, which is simultaneously:
I wonder why they are calling them āmacrosā when they are more like python decorators or java annotations, much less like the syntactic replacement that āmacroā suggests.
I guess the actual reason is the same why Rust calls sum types enums ā itās more important to have succinct word which is clear in context, then to pedantically follow etymology.
Otherwise, I donāt think this is like āannotationsā, more like āannotation processorsā. And given that the primary reason for the processing would be to generate source code (so, not runtime functions, like in the case of pythonās decorators), macros seem defensible to me!
TL;DR āwe donāt know, make a proposal!ā
Thatās not a good representation of what Iām trying to convey here. Itās about how one needs to think about making a proposal.
Is there a way to have type safe macros? I love gleamās error messages and precise inference. Iām afraid macros could lead to what I personally donāt like about elixirās usage of macros: debugging gets harderā¦
Yes! There are lots of macro systems that are type safe.
What do you think of adding module types and functions a la Ocaml? I donāt mean the full system, something in between erlang behaviors and a typed structure.
The full system doesnāt fit well with BEAM due to lack of full program compilation to dedupe generated code, and just typing modules is very limited due it being a weaker version of the record type.
Iām reacting to the headline which made me think there would be any metaprogramming content in the post. Iām pointing out to anyone else interested on that basis that there isnāt any. This is the old āif you want that try submitting an RFC/PEP.ā
Itās certainly not that! I wrote this post in response to people doing that, unproductively so.
@lpil If you implemented the BABLR vm you could gain the ability for gleam to metaprogram itself and also (potentially) every other programming language that exists at the same time
I wouldnāt be surprised if comments of this form were the motivation behind the āCompilers are just programsā section. š
Itās massively easier to say ājust implement this VM and you get all of these benefits!ā than to to change your widely-used compiler to actually do that, deal with all the technical and social ramifications of that change (what does using that VM do to compile times? To runtime performance of compiled programs?), and so on.
Sorry, in my excitement last night I donāt think I offered an adequate response to your technical query. The VM Iām talking about isnāt a runtime VM; instead itās an edit-time VM not unlike tree-sitter. I believe BABLR to be more generally applicable than tree-sitter though, in part due to the lack of error-recovery during parsing
Yeah I know that a one sentence comments probably seems a bit like sniping form the sidelines, but Iād guess that Iāve invested 15,000 hours so far. Four years full time without pay, costing me my lifeās savings. I just completed the sale of my airplane to have a few more months to work.
Now what seemed an impossibly high obstacle now seems more like what you describe ā a set of practical concerns related to how to deploy the tech
My expectation isnāt that parser authors will do the work to support my embedding format, but rather my embedding format will eventually provide so much value that people will fork parsers just to cause them to return data in a standard embedding format. I even expect Zed to adopt it eventually
What you are doing looks interesting but I must say that if you are aiming for wide adoption (spoken as a maintainer of a popular text editor) you need to use something else than Javascript.
LSP and TS are quite popular because itās easy to embed them into very different applications (TS because itās a C lib and can be linked to directly or compiled to WASM, LSP because itās just a JSON IPC API). Javascript is going to be a complete nonstarter for editors like (n)vim, emacs, helix,ā¦
You donāt need JS at all to be able to use CSTML as an embedding format for code. My goal right now is to provide a reference implementation. Once the reference implementation provides a firm target for interoperability I fully expect native implementations to crop up, perhaps as new projects or perhaps as adaptations to existing tools like Tree-sitter or Biome.
BABLR is a parser/grammar layer on top ā a generic system that allows grammars to be defined that emit CSTML parse results. Ideally its results will be immutable, which allows you to think of it as something not unlike a system of XML schema validators.
Hereās where it gets interesting. Once you have an embedded CSTML representation the BABLR VMās APIs are defined in terms of instructions expressed as CSTML, so any scripting language which has access to the embedded form of CSTML can be used to steer execution of a native BABLR core as it chews up text and emits parse trees.
I know this works because even in JS BABLR is implemented as a bootsrapping compiler
I think you have a lot of material and itās all a little unclear how it fits together. I think you need a worked example with detailed docs explaining how you added this on to something else and made it appreciably more awesome than it was before.
Oh absolutely! Iām nearly ready to set the standards in stone. A high degree of accessibility creates lock-in on standards, so Iāve been very cautious in that way thus far.
Just wanted to add that I read the BABLR README, and Iām a huge lang nerd, and I still couldnāt tell what Iām looking at. It sounds like an LSP alternative but beyond that not sure.
Thatās ok! If I can do anything to make it more clear I am at your service. The goal is to provide a narrow waist between tools and languages, and LSP is currently the closest we have. CSTML isnāt modeled on LSP at all though ā itās a new kind of thing which is technologically most closely related to XML and XML schema validators