I intended to write it as a single-pass compiler directly from the syntax tree and targeting C via TCC to enable it to run quickly and on any platform. Unfortunately I also designed a language with top-level execution and nested functions - neither of which I could come up with a good compilation model for if I wanted to preserve the single-pass, no AST, no IR design of the compiler. It’s certainly possible there is a model, but my musings couldn’t figure it out.
I wrote a language with a single pass compiler (compiling to bytecode) and somewhat conventional imperative syntax. The implementation initially went smoothly, and of course the compiler was super fast, with memory allocation only needed to extend the bytecode array. As I extended the language, it became increasingly obvious why compilers for languages with conventional recursive syntax use an AST. I still use the language, because some of my tools are written in it, but I’ve abandoned trying to extend it.
No AST works well for some syntaxes, like Forth and Brainfuck. In retrospect, that class of syntaxes is not for me.
My current language uses an AST. It also has “top-level execution and nested functions” and other nice syntax I learned to like from Haskell and other powerful languages. I didn’t experience any of the pain from the previous language implementation (caused by not having an AST). I even figured out how to compile it into C++ and GLSL. Everything went smoothly, until I tried to get ambitious about doing optimization and partial evaluation. I created a kind of IR, but I botched the design. Next language will have a well designed IR.
I learned that if you want to extend your language to have all the nice things, then you should plan for a multi-pass compiler, it just makes the job so much easier.
In the early language that I implemented (called Gen), I tried to generate byte code directly from the parser, without first converting the code into an intermediate representation. That’s what I meant by “single pass”. This worked fine for statements, but there were tricky cases involving expressions that caused a lot of pain and code complexity.
In Lua 5, expressions are converted into an intermediate representation, of type expdesc. Optimizations can be applied to an expdesc before it is converted into bytecode. That would have solved the problem I encountered in the Gen compiler.
So, by “single-pass” I meant “no intermediate representation”.
In a language like Lua (or Pascal) which has a single-pass recursive-descent parser, the functions that handle each partially-complete syntactic phrase (a block or an expression etc.) keep state representing the translation-in-progress of that phrase. I wouldn’t call this state an IR because it grows with the nesting depth of the program, not with its linear size. It doesn’t represent complete phrases, only work-in-progress.
In Lua’s expression parser you can see that expdesc objects are allocated on the stack, so they can’t represent complete expressions, only nesting. An expdesc contains enough information to represent an intermediate result but not an entire expression. The luaK_ calls emit the code to evaluate the expression on the fly while the expression is parsed.
Yes, I’ve thought about your previous comments on here about the importance of an AST for any compiler as I’ve run into all the many problems with generating code directly from the parse tree. But I’d started inspired by some of the Brian Callahan “Let’s Build a Compiler” blogs. I’ve been trying to decide how much effort I want to put into building an AST for this. One of my goals in using this as a side project was to write everything in such a way that I can always, or almost always, make progress in 15 minutes or an hour of development. So everything is incredibly incremental.
I might have figured out a way around my current nested function generation issue, but I’m sure not having the AST will continue to cause more issues down the road. It’ll influence the type checking in ways I probably don’t want.
Author here. Didn’t expect to see this anywhere. It’s clearly not ready for use by anyone including me. This is just a little playground toy language for me in my free time. My eventual hope/goal is to implement a fantasy console type thing in it and programmed by it. Anyway, the main readme has a combination of actual design goals for the language and current limitations and implementation decisions. It’s up to the reader to discern between the two. ;)
I wasn’t going to comment, but since others have — initially I wasn’t sure if this was a joke, with “features” like “No package manager to distract you” and “Simple type system … everything is an int”.
I mean, I understand it’s in early development, but the website seems to be overpromising and underdelivering. If it were my language I’d have waited until it was farther along before showing it off.
I do think some “features” on here are rather ridiculous e.g. all types are ints, but I do consider the lack of a package manager to be a feature. IMO package management should be delegated to my system package manager or something like Guix/Nix
I think Nix is about as portable between Linux distros as, say, Cargo. AFAIK Nix doesn’t work on Windows, but a new monolingual package manager very well might not either.
Markdown files describing what you need a relatively portable, just that it’s between humans rather than package managers. Some dislike this for good reason, others dont
Parts of the readme are certainly a joke, parts are descriptions of where the implementation is now, parts are things I’d like to do or have done at some point. I certainly wasn’t intending to show it off.
I wrote my own toy language for scripting a game, and I used keywords heavily too (though I did still build an AST and compile phase). I decided to go for the BASIC-like simplicity and readability of something like tell @player "The trap door is stuck." over heavy use of line noise or nested parens. Nice to have company – good luck with yours!
Hey, thanks. I’m curious to see what I think about heavy keyword use after I’ve written more code in the language. I like it in little snippets but my longest program in zinc is like 40 lines - half of which are closing braces.
I see you have some bitmap fonts & tools. I may have to use those.
I wrote a language with a single pass compiler (compiling to bytecode) and somewhat conventional imperative syntax. The implementation initially went smoothly, and of course the compiler was super fast, with memory allocation only needed to extend the bytecode array. As I extended the language, it became increasingly obvious why compilers for languages with conventional recursive syntax use an AST. I still use the language, because some of my tools are written in it, but I’ve abandoned trying to extend it.
No AST works well for some syntaxes, like Forth and Brainfuck. In retrospect, that class of syntaxes is not for me.
My current language uses an AST. It also has “top-level execution and nested functions” and other nice syntax I learned to like from Haskell and other powerful languages. I didn’t experience any of the pain from the previous language implementation (caused by not having an AST). I even figured out how to compile it into C++ and GLSL. Everything went smoothly, until I tried to get ambitious about doing optimization and partial evaluation. I created a kind of IR, but I botched the design. Next language will have a well designed IR.
I learned that if you want to extend your language to have all the nice things, then you should plan for a multi-pass compiler, it just makes the job so much easier.
Lua is a nice example of a language with nested functions and a single-pass compiler.
It depends on your definition of “single-pass”.
In the early language that I implemented (called Gen), I tried to generate byte code directly from the parser, without first converting the code into an intermediate representation. That’s what I meant by “single pass”. This worked fine for statements, but there were tricky cases involving expressions that caused a lot of pain and code complexity.
In Lua 5, expressions are converted into an intermediate representation, of type
expdesc. Optimizations can be applied to anexpdescbefore it is converted into bytecode. That would have solved the problem I encountered in the Gen compiler.So, by “single-pass” I meant “no intermediate representation”.
It’s simpler than you make it sound!
In a language like Lua (or Pascal) which has a single-pass recursive-descent parser, the functions that handle each partially-complete syntactic phrase (a block or an expression etc.) keep state representing the translation-in-progress of that phrase. I wouldn’t call this state an IR because it grows with the nesting depth of the program, not with its linear size. It doesn’t represent complete phrases, only work-in-progress.
In Lua’s expression parser you can see that expdesc objects are allocated on the stack, so they can’t represent complete expressions, only nesting. An expdesc contains enough information to represent an intermediate result but not an entire expression. The
luaK_calls emit the code to evaluate the expression on the fly while the expression is parsed.Yes, I’ve thought about your previous comments on here about the importance of an AST for any compiler as I’ve run into all the many problems with generating code directly from the parse tree. But I’d started inspired by some of the Brian Callahan “Let’s Build a Compiler” blogs. I’ve been trying to decide how much effort I want to put into building an AST for this. One of my goals in using this as a side project was to write everything in such a way that I can always, or almost always, make progress in 15 minutes or an hour of development. So everything is incredibly incremental.
I might have figured out a way around my current nested function generation issue, but I’m sure not having the AST will continue to cause more issues down the road. It’ll influence the type checking in ways I probably don’t want.
Author here. Didn’t expect to see this anywhere. It’s clearly not ready for use by anyone including me. This is just a little playground toy language for me in my free time. My eventual hope/goal is to implement a fantasy console type thing in it and programmed by it. Anyway, the main readme has a combination of actual design goals for the language and current limitations and implementation decisions. It’s up to the reader to discern between the two. ;)
I wasn’t going to comment, but since others have — initially I wasn’t sure if this was a joke, with “features” like “No package manager to distract you” and “Simple type system … everything is an int”.
I mean, I understand it’s in early development, but the website seems to be overpromising and underdelivering. If it were my language I’d have waited until it was farther along before showing it off.
I do think some “features” on here are rather ridiculous e.g. all types are ints, but I do consider the lack of a package manager to be a feature. IMO package management should be delegated to my system package manager or something like Guix/Nix
Coming from a C++ background, I can say a package manager is much, much, much to be desired. A system package manager is not the same thing at all.
Err it’s probably dependent on the preferred workflow. Not sure how the C++ ecosystem works but I vastly prefer just using apk (sometimes guix) in C
This means that your project will be unportable not only between operating systems, but between distros.
I think Nix is about as portable between Linux distros as, say, Cargo. AFAIK Nix doesn’t work on Windows, but a new monolingual package manager very well might not either.
Markdown files describing what you need a relatively portable, just that it’s between humans rather than package managers. Some dislike this for good reason, others dont
Parts of the readme are certainly a joke, parts are descriptions of where the implementation is now, parts are things I’d like to do or have done at some point. I certainly wasn’t intending to show it off.
I wrote my own toy language for scripting a game, and I used keywords heavily too (though I did still build an AST and compile phase). I decided to go for the BASIC-like simplicity and readability of something like
tell @player "The trap door is stuck."over heavy use of line noise or nested parens. Nice to have company – good luck with yours!Hey, thanks. I’m curious to see what I think about heavy keyword use after I’ve written more code in the language. I like it in little snippets but my longest program in zinc is like 40 lines - half of which are closing braces.
I see you have some bitmap fonts & tools. I may have to use those.