Regarding the parenthesis-as-expressions, does this make control blocks like if, while, etc, regular commands, semantically, of the form if EXPRESSION BLOCK, where the expression just follows the normal expression mode rules instead of explicitly requiring parenthesis as a grammar rule.
I wonder if you could push that further? It might lead you into a remarkably elegant Lispy or Haskellesque territory, and give you fully programmable control flow for free. Very possible this is already the case and I’m not seeing it yet.
You’re in the verge of ^ as quotation, there, which would be incredible.
I imagine keeping all of this compatible-ish with Bash is always a constraining factor, so if these are made impossible by that I’d be super curious as to why.
Yes great points! I’m glad someone is paying attention :)
For the first question, if and while are a special keywords basically because they are in shell, and I implemented them a long time ago. But I posted this same question on Zulip today under this post!
So I am wondering if it is confusing that the syntax is the same, but they mean different things. But in practice I’m not sure if there are any real consequences, other than the fact that you can’t redefine if as a function. And I don’t really want that. Testing and feedback is appreciated!
And yes there is an inconsistency in the punctuation. I documented that here:
But the basic reason is that we already solve the problem of parsing $(echo hi) and the closing paren, which is actually quite difficult because parens are used in so many places, and can be unbalanced in case statements. (The AOSA chapter on bash which I refer to talks about this problem.)
Parsing $[echo x] and ${echo hi} is actually even more difficult, because neither of them are operator characters. Unquoted [] can be globs or the test builtin, and unquoted {} can be brace expansion or brace groups.
So I avoided those, and made ^(echo hi) consistent with $(echo hi). (And we also have @(echo hi) for split command sub.) The syntax is basically consistent with shell. If you think about it, shell also has $(echo hi) for command sub and { echo hi; } for blocks.
Originally I did specify that command sub was $[echo hi], because [] was supposed to be for “words” and () was supposed to be for expressions. But that collided with reality!
So it is a wart, and a documented one, but I’m actually OK with it because I think those unevaluated expressions will be very rare in real code. They are probably only going to be used for framework and library code (which I also hope will be rare!)
I did spend a lot of effort trying to make all the sigils and punctuation consistent, and wrote a whole doc about it!
The appendix is out of date, I just pushed a change [1], although I’m still confused where you got a lot of those. For example () isn’t a command; it’s a type arg list to a command as shown in the blog post.
Expression mode is like Python / JavaScript! Generally this code has very few sigils and “sigil pairs”. It looks like
const x = f('mystr', [], myvar)
{} [] () mean what they do in Python / JS – dict, list, parenthesized expression
we also have %(foo bar) which is identical to ['foo', 'bar']. It’s like Perl’s qw//. Words are unquoted.
Then in command mode:
Word splitting is explicit rather than implicit. So we need:
Explicit splicing of array variables with @myarray
Split command sub with @(command sub)
These are analogous to the two above
We also have expression sub, which is $[1 + 2].
You can pass { echo hi } as a trailing block arg, which is consistent with shell’s brace groups.
Everything else should be quite rare. The inconsistent unevaluated expressions are a wart, but as mentioned very rare. Also, that part is less implemented (though everything in the Tour is verified to work).
Does that make sense?
%{} and @{} don’t have any obvious meaning I see. I guess you could argue that @{myarray} should mean splicing, but it’s superfluous considering the meaning of $myvar vs ${myvar} (which is purely syntactic).
You just introduced me to uftrace, thanks!
Cool, yeah it came in handy this release as a substitute for gdb / CLion! Somehow I’m unable to productively use GDB’s interface for many bugs …
I was using CLion a lot earlier this year but switched PCs and didn’t install it again. Surprisingly, uftrace is a nice CLI substitute in some cases.
Originally I had used it for performance reasons, but it’s also good for debugging. I meant to write a a post tagged #praise about it, which I mentioned here: http://www.oilshell.org/blog/2020/01/blog-roadmap.html
Simply counting function calls can be a great for profiling real systems. Particularly recursive descent parsers. And uftrace is really good at that.
I also haven’t written properly about re2c, although I keep mentioning it, and it’s been great for ~4 years now
This continues to be really fun to see develop.
Regarding the parenthesis-as-expressions, does this make control blocks like
if,while, etc, regular commands, semantically, of the formif EXPRESSION BLOCK, where the expression just follows the normal expression mode rules instead of explicitly requiring parenthesis as a grammar rule.I wonder if you could push that further? It might lead you into a remarkably elegant Lispy or Haskellesque territory, and give you fully programmable control flow for free. Very possible this is already the case and I’m not seeing it yet.
I imagine you’ve caught this, but:
sort of asks to be rewritten as:
You’re in the verge of
^as quotation, there, which would be incredible.I imagine keeping all of this compatible-ish with Bash is always a constraining factor, so if these are made impossible by that I’d be super curious as to why.
Yes great points! I’m glad someone is paying attention :)
For the first question,
ifandwhileare a special keywords basically because they are in shell, and I implemented them a long time ago. But I posted this same question on Zulip today under this post!https://oilshell.zulipchat.com/#narrow/stream/202008-oil-documentation/topic/Winter.20Blog.20Backlog.3A.20Recent.20Progress
So I am wondering if it is confusing that the syntax is the same, but they mean different things. But in practice I’m not sure if there are any real consequences, other than the fact that you can’t redefine
ifas a function. And I don’t really want that. Testing and feedback is appreciated!And yes there is an inconsistency in the punctuation. I documented that here:
https://www.oilshell.org/release/0.9.5/doc/warts.html#two-different-syntaxes-for-block-and-arglist-literals (some old syntax here; just corrected at HEAD)
But the basic reason is that we already solve the problem of parsing
$(echo hi)and the closing paren, which is actually quite difficult because parens are used in so many places, and can be unbalanced in case statements. (The AOSA chapter on bash which I refer to talks about this problem.)Parsing
$[echo x]and${echo hi}is actually even more difficult, because neither of them are operator characters. Unquoted [] can be globs or thetestbuiltin, and unquoted{}can be brace expansion or brace groups.So I avoided those, and made
^(echo hi)consistent with$(echo hi). (And we also have@(echo hi)for split command sub.) The syntax is basically consistent with shell. If you think about it, shell also has$(echo hi)for command sub and{ echo hi; }for blocks.Originally I did specify that command sub was
$[echo hi], because[]was supposed to be for “words” and()was supposed to be for expressions. But that collided with reality!So it is a wart, and a documented one, but I’m actually OK with it because I think those unevaluated expressions will be very rare in real code. They are probably only going to be used for framework and library code (which I also hope will be rare!)
I did spend a lot of effort trying to make all the sigils and punctuation consistent, and wrote a whole doc about it!
https://www.oilshell.org/release/0.9.5/doc/syntax-feelings.html (feedback welcome)
Let me know if you see any other inconsistencies :)
(): is a command$():$means string / scalar,()is a command^():^means quoted,()is a block@():@means array / splice,()is a command%():%means unquoted,()is an array literal (sequence of words){}: brace expansion or brace groups${}:$means string / scalar,{}is variable substitution (its own insanity)^{}:^means quoted,{}is … nothing? It was an expression.@{}: is a syntax error%{}: is … nothing?[]: is a list (sequence in the docs)$[]: is … nothing?^[]:^means quoted,[]is an expression@{}: is a syntax error%{}: is … nothing?Even with oil and the docs both open, I don’t feel confident with my above listing.
Is there a table somewhere? This feels like Perl-level inconsistency.
The appendix is out of date, I just pushed a change [1], although I’m still confused where you got a lot of those. For example
()isn’t a command; it’s a type arg list to a command as shown in the blog post.The best overview at the moment is the tour:
https://www.oilshell.org/release/latest/doc/oil-language-tour.html
Here’s a short way to think about it. Let’s start with the following axioms from shell:
$varand${myvar}$(command sub)$(( ))and bash’s[[and(({ echo 'brace group'; }is{ echo 'brace group' }in OilThen in Oil, you have command vs. expression mode:
https://www.oilshell.org/release/latest/doc/command-vs-expression-mode.html
Expression mode is like Python / JavaScript! Generally this code has very few sigils and “sigil pairs”. It looks like
{} [] ()mean what they do in Python / JS – dict, list, parenthesized expression%(foo bar)which is identical to['foo', 'bar']. It’s like Perl’sqw//. Words are unquoted.Then in command mode:
@myarray@(command sub)$[1 + 2].{ echo hi }as a trailing block arg, which is consistent with shell’s brace groups.Everything else should be quite rare. The inconsistent unevaluated expressions are a wart, but as mentioned very rare. Also, that part is less implemented (though everything in the Tour is verified to work).
Does that make sense?
%{}and@{}don’t have any obvious meaning I see. I guess you could argue that@{myarray}should mean splicing, but it’s superfluous considering the meaning of$myvarvs${myvar}(which is purely syntactic).[1] https://github.com/oilshell/oil/commit/7adffce97ba8619f9fb866b1d185c1f0d6cd45b3
Not really, no. That’s why I asked if there was a table.
oil -n -c '…'mostly.This table is updated (and will be published with the next release).
https://github.com/oilshell/oil/blob/master/doc/syntax-feelings.md
You have to take into account the valid contexts (command, expression or both), and what’s inside (either commands, expressions, or words).
Thanks mate; I gave a look and it’s a lot more clear!
Am I blind, though, or is it missing normal ’ol
$variable/${variable}?Another thing to note that
%does not mean hash, like it does in Perl.$and@do mean string and array respectively, so this could be confusing.Oil doesn’t need any sigil for hash, because expression mode is different than command mode. The sigils are used mainly in command mode.