@ for splice feels weirdly Perl. JS and Python users would know … (spread syntax) or * (unpacking operator)
It feels more Lisp than Perl to me. In Perl, @foo is just the list stored in variable foo, not that list spliced into any outer context. Within a Lisp quasiquotation, on the other hand, ,@foo means to splice in the list stored in foo.
Good point, I will explain :d, but : is sort of a pseudo-sigil that means “this is a variable name that will be modified. Instead of shell’s read x, you do read :x which I think is more clear.
It’s bash syntax. I wanted to change it to c'\n', but that turned out to be hard to parse in command mode, and it also introduces complications with multiline strings. c'foo' already means something in shell. It might be possible to get rid of, but I feel like it’s a lot of complexity and explaining.
Oil supports all the shell operators, but I felt :- is the one that’s still useful. I somewhat agree about ?? and that is on Github somewhere. But overall I do feel like there’s value in keeping OSH+Oil small, not just Oil small.
The @ kind of comes from shell itself with "$@", and PowerShell / Perl. We can’t use * because that means glob, and ... is going to be used for multiline commands.
Brace expansion and globs are both part of the word language, and we don’t change it. I don’t see the problem; they’re “time tested” at this point.
proc means “procedure” or “process”. Added to the wiki.
Added to the FAQ.
Yes if (x) and while (x) are special variants of the shell if and while. There might be a for () later – it would make sense but isn’t strictly necessary now.
! inverts an exit code while not inverts a Python-like boolean. Added to the FAQ.
Not sure exactly what this means, but the case syntax is quirky with ;; and (glob) basically because of shell legacy. I decided not to change it too much.
This is a syntax error. Try it in Oil! I think you might need to get to “command vs. expression mode”. For another analogy, ! is part of command mode, and not is part of expression mode.
There should be functions, but I haven’t fully defined them yet! I think they will be built ON TOP of “procs”.
True False None are compatible with Python. I will update the doc. Oil’s expression syntax is largely derived from Python, with a few exceptions. I’m working on an “Oil vs Python” and “Oil vs Shell” doc.
Hm ~= instead of ~== is possible, but I didn’t want it to get confused with Perl’s regex matching operator. The negated form !~= or ~!= is sort of an awkward problem though. It could change.
Yeah there is some argument for this, but I think it’s OK to have one form that mutates and one form that creates a new object.
Good question, I’ve debated read --json and write --json. That’s too long to answer here, but join me on Zulip and we can talk about it :) (link on home page)
Yes OSH and Oil are able to be used in the same file, but you’re not really supposed to. You could put shopt -- set oil:all halfway down the file if you want, and start using Oil syntax, but it’s better to keep it separated by file.
They are really the same interpreter, parameterized by shopt options. That is what the “runtime” section of the doc is supposed to convey, but I can see why that is not super clear now.
This was very helpful for me, as someone who’s taken a look at osh/oil from time to time, but never gone that deep. A few observations/requests to say certain things explicitly.
In the list section, you mention arrays and lists. Are they different? Are dict keys always strings? Can they be quoted strings with underscores or meta chars? Are expressions always in parens?
Structurally, I wonder if it might make sense to move some examples to the top to give people more of a feel, before talking about words, and other things that might matter less to someone who lands on the page without familiarity with the project.
Thanks, this is great feedback, I will update the doc. (Although some of it will have to go on linked docs to keep the length down.)
Yes good point array vs. list is confusing. Oil is like Python – it only has lists. I sometimes use “array” to mean “list of strings” but I think I should settle on one term. I want to keep consistency with Python’s terminology, but “list” is sort of a bad name, since it reminds people of linked list? JavaScript uses “array” so maybe I should adopt that terminology.
JS: array and “object” (object isn’t good)
Python: list and dict
Oil: array and dict? Or maybe array and map? Not sure if that’s a good idea.
The dict keys have to be strings, unlike Python. The rule is the same as in JavaScript, except with the -> operator instead of . (dot)
d->key is the same as d['key']
If you have special chars, do d['foo+bar'], since d->foo+bar is parsed like addition.
Yes, good point about examples. I got other feedback that the “word/command/expression” stuff was too abstract, so I will try to reduce / de-emphasize it. I think it makes sense for the END of the doc rather than the beginning.
I will update the doc but let me know if you more feedback!
I made another pass over this. There are a few TODOs but it lists almost everything.
Let me know what you think! The language is still open to suggestion, especially from people who try it on real use cases. Actually almost everybody who has tried it has ended up influencing the language in some way (e.g. Till Red, Raphael Megzari recently)
It was nice to read. While Oil often appears on this site, I didn’t really understand what the language is trying to do until now. It takes the conveniences of shell programming, not having to escape every string every time. And adds JSON data structures on top.
Yes that’s one way to describe it! JSON is an almost universal interchange format, and a shell that has JSON support should also have JS/Python-like data types and expressions to complement it.
I have a draft of a blog post called “Essential Features of the Oil Language” which describes these as the four main features:
Python-like data types and expressions
Egg Expressions (Oil Regexes)
Abstraction With Shell-like procs and Ruby-like blocks
Interchange formats like JSON and QTT (quoted, typed tables)
Safe handling of user-supplied, untrusted data (QSN, etc.)
Eliminate quoting hell (arrays and splicing, mentioned in the doc)
Static Parsing for better error messages and tooling
So I’d say those 8 things are a good summary of what Oil’s trying to do! It’s taking awhile but I don’t see any one else doing a graceful upgrade of shell.
There is https://www.nushell.sh/. It doesn’t try to be a scripting language, but it has similar goals as a shell. The main difference I would say is that they extend the pipe to support types, instead of splitting commands and functions as Oil does. Both have pros and cons.
myls -q | where [size > 10] # structured data that's then filtered by 'where' builtin
but it’s a little far in the future. My thinking is that myproc [a, b] is actually a lazy argument list, and I can parse it with shopt --set parse_bracket (analogous to how we use shopt --set parse_paren parse_brace to make if (x) { ... } work.
That is all hidden under bin/oil, so you don’t need to remember the names. But that is the underlying mechanism for changing the syntax, and gradually migrating OSH to Oil.
Here are all the questions / bikesheds that popped in my mind as I took the tour!
Why an explicit
setvarkeyword?:din the first write JSON example is surprising… and never explained laterWhat prevents removing this unfortunate syntax collision?
How many other shell operators are there beyond
:-?And why add them since Oil has a modern expression language?
I guess I miss the
??/?:operator.@for splice feels weirdly Perl. JS and Python users would know...(spread syntax) or*(unpacking operator)The brace expansion mini-language undermines the promise of the word/command/expression languages being comprehensive.
Why
procoverfunorfuncorfunction?Why
->over.? Neither JS nor Python use->. Does anything outside of C and C++?Is the
()expression form specific forwhileandif?Why both
!andnot?Is the
()form incasedifferent than forwhileandif?🤔 What does
if (not try myproc)do then?Oh! There are functions! (I clicked on the link in the Ruby-like Blocks section.)
Compatibility with what?
Why
~==instead of~=, keeping all comparison operators two characters?appendandextendgo away with[1] + [2]or[1, @[2]]:dshowed up again. My guess is that it’s a symbol (ala Ruby) and thatjsoncan reach into the scope.Why isn’t
jsona pair of functions? Or isn’tread --jsonalaread --qsn?Wow, the Oil language is exciting! A cleaned up shell, sign me up!
Please add versioning to it now, before it’s too late. 🙏🏾
It feels more Lisp than Perl to me. In Perl,
@foois just the list stored in variablefoo, not that list spliced into any outer context. Within a Lisp quasiquotation, on the other hand,,@foomeans to splice in the list stored infoo.Great feedback! I started a FAQ here: https://github.com/oilshell/oil/wiki/Oil-Language-FAQ
And here are some more answers.
Good point, I will explain
:d, but:is sort of a pseudo-sigil that means “this is a variable name that will be modified. Instead of shell’sread x, you doread :xwhich I think is more clear.I mentioned the
$''issue here: https://www.oilshell.org/release/0.9.0/doc/warts.htmlIt’s bash syntax. I wanted to change it to
c'\n', but that turned out to be hard to parse in command mode, and it also introduces complications with multiline strings.c'foo'already means something in shell. It might be possible to get rid of, but I feel like it’s a lot of complexity and explaining.Oil supports all the shell operators, but I felt
:-is the one that’s still useful. I somewhat agree about??and that is on Github somewhere. But overall I do feel like there’s value in keeping OSH+Oil small, not just Oil small.The
@kind of comes from shell itself with"$@", and PowerShell / Perl. We can’t use*because that means glob, and...is going to be used for multiline commands.Brace expansion and globs are both part of the word language, and we don’t change it. I don’t see the problem; they’re “time tested” at this point.
procmeans “procedure” or “process”. Added to the wiki.Added to the FAQ.
Yes
if (x)andwhile (x)are special variants of the shellifandwhile. There might be afor ()later – it would make sense but isn’t strictly necessary now.!inverts an exit code whilenotinverts a Python-like boolean. Added to the FAQ.Not sure exactly what this means, but the case syntax is quirky with
;;and(glob)basically because of shell legacy. I decided not to change it too much.This is a syntax error. Try it in Oil! I think you might need to get to “command vs. expression mode”. For another analogy,
!is part of command mode, andnotis part of expression mode.There should be functions, but I haven’t fully defined them yet! I think they will be built ON TOP of “procs”.
True False Noneare compatible with Python. I will update the doc. Oil’s expression syntax is largely derived from Python, with a few exceptions. I’m working on an “Oil vs Python” and “Oil vs Shell” doc.Hm
~=instead of~==is possible, but I didn’t want it to get confused with Perl’s regex matching operator. The negated form!~=or~!=is sort of an awkward problem though. It could change.Yeah there is some argument for this, but I think it’s OK to have one form that mutates and one form that creates a new object.
Good question, I’ve debated
read --jsonandwrite --json. That’s too long to answer here, but join me on Zulip and we can talk about it :) (link on home page)What do you mean by versioning?
Mate, cheers for the extensive explanations and follow-up! 🙇🏽♀️
Oh! I think I misunderstood. Are Oil and OSH both able to be used in the same ifle?
I mean a way to indicate what version of Oil is used in a file.
shopt oil:2021opts the file into the current dialect. But the inevitable major changes come, leave an out withshopt oil:2048.Yes good point! bash has shopt -s compat44 for compatibility with bash 4.4, etc.
I wouldn’t add that for individual options, but I think adding it for the groups makes sense: https://github.com/oilshell/oil/issues/973
Yes OSH and Oil are able to be used in the same file, but you’re not really supposed to. You could put
shopt -- set oil:allhalfway down the file if you want, and start using Oil syntax, but it’s better to keep it separated by file.They are really the same interpreter, parameterized by
shoptoptions. That is what the “runtime” section of the doc is supposed to convey, but I can see why that is not super clear now.Great feedback, thanks!
This was very helpful for me, as someone who’s taken a look at osh/oil from time to time, but never gone that deep. A few observations/requests to say certain things explicitly.
In the list section, you mention arrays and lists. Are they different? Are dict keys always strings? Can they be quoted strings with underscores or meta chars? Are expressions always in parens?
Structurally, I wonder if it might make sense to move some examples to the top to give people more of a feel, before talking about words, and other things that might matter less to someone who lands on the page without familiarity with the project.
Thanks, this is great feedback, I will update the doc. (Although some of it will have to go on linked docs to keep the length down.)
->operator instead of.(dot)d->keyis the same asd['key']d['foo+bar'], sinced->foo+baris parsed like addition.I will update the doc but let me know if you more feedback!
I made another pass over this. There are a few TODOs but it lists almost everything.
Let me know what you think! The language is still open to suggestion, especially from people who try it on real use cases. Actually almost everybody who has tried it has ended up influencing the language in some way (e.g. Till Red, Raphael Megzari recently)
It was nice to read. While Oil often appears on this site, I didn’t really understand what the language is trying to do until now. It takes the conveniences of shell programming, not having to escape every string every time. And adds JSON data structures on top.
Yes that’s one way to describe it! JSON is an almost universal interchange format, and a shell that has JSON support should also have JS/Python-like data types and expressions to complement it.
I have a draft of a blog post called “Essential Features of the Oil Language” which describes these as the four main features:
https://oilshell.zulipchat.com/#narrow/stream/266575-blog-ideas/topic/Five.20Essential.20Features.20of.20the.20Oil.20Language
On the compatible side, we have Four Features of the OSH Language:
So I’d say those 8 things are a good summary of what Oil’s trying to do! It’s taking awhile but I don’t see any one else doing a graceful upgrade of shell.
Thanks for the feedback!
There is https://www.nushell.sh/. It doesn’t try to be a scripting language, but it has similar goals as a shell. The main difference I would say is that they extend the pipe to support types, instead of splitting commands and functions as Oil does. Both have pros and cons.
Yup I’ve looked at almost all alternative shells and made a huge list of them! https://github.com/oilshell/oil/wiki/Alternative-Shells
I would like to add something like:
but it’s a little far in the future. My thinking is that
myproc [a, b]is actually a lazy argument list, and I can parse it withshopt --set parse_bracket(analogous to how we useshopt --set parse_paren parse_braceto makeif (x) { ... }work.That is all hidden under
bin/oil, so you don’t need to remember the names. But that is the underlying mechanism for changing the syntax, and gradually migrating OSH to Oil.