1. 28
  1.  

  2. 8
    let name = 'foo bar'
    await $`mkdir /tmp/${name}`
    

    It seems it has shell escaping built-in. Nice.

    1. 2

      Yes, it does. 🙌

        1. 6

          Template literals is a JS feature but in the example above what directory would be created? foo or "foo bar"? I think the OP meant that with “built-in shell escaping” the example will create "foo bar" directory without the need for you to wrap spaces with double quotes.

          1. 4

            Yes, you are right. Zx escapes every argument which comes from ${}.

      1. 6

        Waiting for @andyc comments on JS vs. Shell.

        1. 7

          I think there’s definitely a place for internal DSLs in every language, but we also need a new Unix shell :)

          http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#what-about-this-shell-like-library-embedded-dsl

          1. 7

            I want to chime in with a somewhat different opinion. Not entirely contrary, but with some more detail that I hope will be useful for people to think about the tradeoffs.

            To start, let’s agree: your link to your site mentions that the shell is a low-level component that needs to be available with few dependencies. There are various things you want to do with shell scripts with absolute minimum dependencies, like early boot scripting or user-level scripts to install your basic environment dependencies (eg. to install another shell!). And we absolutely should have a better shell than Posix Shell or Bash for this stuff.

            However, I think most shell scripts don’t need to be so spartan, and for these cases an embedded shell language can be much more powerful.

            Note that not all embedded shell DSLs are the same or even in the same category. This zx looks like it’s in the category of Plumbum for Python or Scsh for Scheme. With these, you have a library to easily write shell commands or pipelines inside a host language, but I wouldn’t consider these “shells” as much as “shell libraries” because they really aren’t suited for interactive use. In other words, nobody really fires up Python with Plumbum (or Scsh) to play around in the shell and find the solution, then copy/paste it to a script. Instead, they start directly at the script level. Thus these libraries don’t capture the full shell scripting process that people commonly use. This is why “just start with python rather than writing a shell script” often doesn’t happen. You really want that interactive first step for many things, and translating interactions to another language really seems to be a hurdle (especially when eg. you are interacting with Bash but some features, eg. maybe globbing, don’t translate directly or easily to the embedded DSL).

            Other embedded shells, like Rash in Racket (my project) or Xonsh in Python, are a much more thoroughly modified DSL designed with live shell interactions in mind. For example, they use custom parsers to have an interactive-friendly terse syntax more like Bash while being able to escape to the host language (though the quality and flexibility of the embedding varies). This much more fully captures the spirit and lifecycle of shell programming – you can iterate with commands live in a repl, copy them to a script, and grow the script from there. However, with an embedded shell DSL, when you start growing the script you have the whole host language available. Not only does this include core language design that’s done poorly in popular shells (eg. control flow, error handling, data structures, etc), but you get the entire language package ecosystem (this is where Xonsh really shines) and any advanced features that a stand-alone shell language wouldn’t have (eg. Rash gets to use Racket’s macro system, interoperates with Typed Racket and other languages embedded in Racket, has first-class delimited continuations, etc). This allows scripts to grow gradually (without a full rewrite) not only past the “Bash has impoverished data structures and is full of landmines” stage where most shell scripts are scrapped, but also past the “this part would be more {efficient, complete, clean, etc} if done natively in a general-purpose language” phase where some shell scripts need to graduate to become “real” programs.

            Current embedded DSL shells have various shortcomings. Eg. while Rash is designed to be a language for live interaction, it desperately needs a new line editor to even begin to compete with Zsh or Fish on interactive completion, history management, or even basic command editing. (Completion is a common problem for shells, and there have been some discussions that @andyc has hosted about building re-usable tooling for it.) Embedded shells tend to lack a lot of polish because they are all just hobby or grad school projects. (If I were financially independent I could focus on Rash full time for a few years and really polish it up, but alas! Maybe after I graduate and work in industry for a while.) Xonsh is probably the most polished for interactive use, though I think it’s weaker than some other embedded shell DSLs in other respects. (Not to disparage Xonsh – it is one of the oldest in its class and later ones had the benefit of hindsight.)

            Despite their current issues, I think the brightest future for shell programming is high quality embedded shell DSLs. In the long term, or at least in some idealized long term where sufficient manpower is given to both stand-alone shells and embedded shells, stand-alone shell languages won’t be able to compete with embedded shells in terms of features, libraries, customizability, etc.

            (Just to be clear about the thread’s original topic, I think zx looks cool an useful, but it is clearly not an attempt to be this higher level of embedded shell DSL I’ve discussed here.)

            1. 2

              Hm interesting, I do see your point about the difference between between shell libraries and internal shell DSLs. Although I use neither of them so I guess I don’t have much to say about them!

              I also haven’t encountered them “in the wild” very much; when I look at open source projects I tend to see shell scripts. For example almost all the docs and examples for Github Actions or Travis CI is shell-in-YAML.


              As for whether embedded DSLs are the brightest future of shell programming – I think it depends on the definition of “shell”. It’s become very clear to me that not everyone uses it the same way or thinks of it the same way.

              Some people mean “an interactive REPL” and actually Jupyter notebooks more or less qualify as a “better shell” for them.

              For me, shell is the lowest common denominator between languages, and it deals directly in kernel concepts like byte stream and processes, without abstraction.

              I use it to glue together Python and R, R and C++, Python and JavaScript, etc.


              I had a long exchange with @xiaq, designer of Elvish a few days ago, on the architectural ideas of the Perlis-Thompson Principle and the “narrow waist”:

              https://lobste.rs/s/ww7fw4/unix_shell_history_trivia#c_epslyn

              This probably needs to be in blog posts and not lobste.rs comments, but the claim is that each language is defined by its core data structures.

              • POSIX sh: everything is a string
                • bash: everything is a string, with arrays as secondary
                • Oil: everything is a string, with JSON-like data structures as secondary
              • Elvish: Everything is a JSON-like value, which are primary. These values can travel directly over pipelines without serialization.
              • Nushell: Everything is table.
              • Rash: I haven’t used it, but presumably everything is a cons cell like in Racket?

              If you define shell as the “lowest common denominator”, then my claim is that sh/bash/Oil is actually the lowest common denominator between an Elvish, Nushell, and Rash script :)

              I’m not necessarily making a claim about which is better or which is easier to use. I’m just saying it’s a fact of software that there are divergent mental models and meta-models, and the lowest common denominator will always be literal operating system concepts like byte streams and processes. At least until something else supplants Unix as the narrow waist of essentially all computing :)

              So that is my view on the future of shell programming. Basically computing is being applied to more areas, and used by more people. So it’s getting more diverse and will never consolidate. That even includes more shell-like languages that needs to be bridged :) And it includes C++, Rust, and Zig. And Matlab and Julia, etc.

              This post has a lot of color on those views, with a bunch of slogans and fallacies (shell is the language of diversity and heterogeneity; processes aren’t slow; programming with strings isn’t slow, etc.)

              http://www.oilshell.org/blog/2021/07/blog-backlog-1.html


              As an example of the difference, I’d ask the same question I did of Elvish. How do you parallelize Rash functions with xargs -P 8, or is that not idiomatic?

              I have another “alternative shell challenge here”: https://www.oilshell.org/blog/2020/02/good-parts-sketch.html#the-alternative-shell-challenge

              This is not to say that other definitions of “shell” aren’t valid. I’m only saying that we do need exactly this kind of shell. We need a narrow waist that’s a tiny layer on top of the kernel.

              1. 2

                Lack of adoption so far doesn’t mean embedded shell DSLs aren’t a good idea. Every good idea that has become popular (electricity, personal computers, the internet) initially had no users.

                Lack of adoption of embedded shell DSLs may perhaps be because there are relatively few of them, they are relatively unknown, most of them are relatively immature, and people don’t trust that they will be maintained in 5 years? These are just growing pains of a new technology. If there is continued work on high quality shell DSLs we can move past this phase.

                That said, I have seen libraries like Plumbum used in various places, and Rash has a few users (including obviously myself). I haven’t seen anybody using Oil shell, but that doesn’t mean it’s a technological dead end, or even that there really aren’t people using it. Over time I hope to see more people using Rash and Oil.


                I agree that the future of “shell” is divergent, and there is room for Oil, Rash, Jupyter notebooks, and more. We probably need better ways of discussing “shells” to help people understand the differences and tradeoffs between these different visions, and which one(s) are more suitable to their goals.


                Rash: I haven’t used it, but presumably everything is a cons cell like in Racket?

                Everything is not a cons cell in Racket. Racket of course has cons cells, but also dictionaries, and various kinds of atomic data (numbers, strings, …), and procedures, and user-definable structures that can be properly encapsulated with APIs, generic interfaces, and higher order contracts… So values in Racket/Rash can be whatever kind of rich data structure you want. And of course Racket has libraries for JSON, CSV, XML, etc, so you can communicate with things that read or write those formats.

                Now, when sending data to a subprocess you probably care about serializability, and you can’t serialize things like procedures or continuations. However, Rash pipelines can have subprocesses as well as Racket procedures, so pipelines with Racket procedures don’t have that limitation. Also, Rash has arbitrary user-definable “substitutions”. IE Bash has substitutions like $(...) (replace the argv element with the result of this code), and <(...) and >(...) (replace the argv element with a path to a file descriptor for this pipeline). One of the demos of user-definable substitutions I made is closure substitution, where the program gets a string that is a path to an executable script written on-the-fly that connects back to the original script via socket so you can use Racket procedures with programs like find -exec.

                (Substitutions are yet another language feature that should be user-definable and extensible. Maybe someone wants temporary directory substitution, or mount point substitution where you mount a virtual file system for the duration of a command, or… you could take this to silly extremes, but user-extensibility is important. This is basically the same argument that a language should have a good macro system so a user can build a for-each loop on top of a regular for loop without waiting for language designers to drag their feet. Of course, the main difference between substitutions in Rash and just using a procedure to compute what the arguments should be is that substitution objects get a clean-up hook.)


                I agree that a “narrow waist” shell that just supports the raw OS primitives should exist. However, I disagree that a large percentage of what I consider “shell scripting” should be done in that shell. Abstraction is powerful, convenient, and helpful. Just like most programming has gradually moved to higher-level programming languages with only specific things remaining firmly in the domain of “use assembly” or “use C”, I think many visions of shell scripting are better served by a powerful, generous language rather than a “lowest common denominator”.

                To me, “shell languages” are mostly about live interactions and automating things. Eg. getting programs to work together, finding ways to do tasks at the command line then saving that in a script if you want to do it again, etc. The idea of using the lowest common denominator tool, or the purity of encapsulating specifically the “narrow waist” of Unix, are generally of little concern or even anti-values. If higher-level tools with richer abstractions make it easier to live code, automate, and maintain automation, they are probably better choices most of the time. I want to use a powerful tool that lets me easily and powerfully interact with my environment, and that lets me write a script and get on with other things (but then also lets me come back and grow my script when I realize that I need to improve that automation).

                Live coding and automation aren’t my whole view of shell programming. Eg. I think it’s also often for quick prototyping (which I think is Rash is well suited to), and about making one-liner alias-like command wrappers (which frankly I don’t use Rash for because it has a slow start-up), and lots of other things, some of which would definitely be better in something like Oil rather than something like Rash, or perhaps in a Jupyter notebook, etc. But I think things like automation and prototyping are very significant chunks of shell programming that benefit more from rich, flexible, powerful languages rather than languages that are trying to be a lowest common denominator. And I think embedding a shell into a host language that is already designed to be flexible and powerful with strong abstraction mechanisms is the easiest and most reasonable way to have all of that in a shell, and the only way to provide a gradual upgrade path for scripts that over time need to become more fully featured programs suited to a general-purpose language.

                The live coding part is still in many ways better in Zsh or Fish than in Rash, but that’s for lack of time and effort into a line editor, not because writing command completions and interactive widgets in Zsh is a better idea than writing them in Racket. I think that should be pretty obvious to anybody who has tried to write shell completion functions or Zsh widgets. One beauty of embedding in Racket is that eventually one could write an emacs-like line editor that can have editing functions (including completions, interactive searching/narrowing widgets, etc) written in Racket, or any Racket-hosted language, including DSLs tailored specifically to writing completions or interactive widgets.


                How do you parallelize Rash functions with xargs -P 8?

                You can use closure substitution (mentioned above), along with Racket’s (honestly somewhat poor) means of parallelization.

                Eg. xargs -P 8 (closure-substitute my-racket-function)

                Or you could pipeline the data directly to a racket function that handles parallelization rather than going through xargs. You could write it directly or make an xargs-like function inside Racket that parallelizes a function that you pass it, reading input from a pipeline line by line and processing it. I would probably do that instead of using an xargs subprocess to paralellize code that I’m already writing in Rash/Racket.

                Eg. data-generating-command |> parallel-xargs-in-racket #:jobs 8 my-function

                That said, parallelizing a shell function with xargs -P 8 is something that, I believe, most shells including Bash and friends can’t do. Does Oil have an xargs builtin or something like a fork server substitution that sends xargs a path to a program that forks the shell to service subprocess calls?

                (A user could write a fork server substitution in Rash, though I think Racket isn’t necessarily fork-safe in all situations. I’ve used fork in Racket programs, but I don’t actually recommend using fork in Rash. Actually, I think using fork directly is generally an anti-pattern, though sometimes it is the only way to do certain things in Unix. If you really need to fork, Rash may not be the right tool. Again, this is about different visions of what shell programming is – in any shell script I’ve ever written or worked with, use of fork in the shell was an implementation detail, not a critical necessity. For those to whom “shell scripting” is all about using fork in specific ways, Oil is probably the better fit. That said, in a fork-safe language, or if Racket exposes a user-level way to fork with a guarantee of safety, an embedded shell DSL might still be the right choice there.)

                Oh, now I look more closely at your post and see your $0 dispatch pattern. Yes, Rash can do that. And… any language where you can access the full argv parameter and get the path to the current executable should be able to. However, $0 dispatch can’t capture dynamic state like closure substitution can. Closure substitution allows you to use higher-order and stateful functions with things like xargs and find.

                I think Rash performs quite well against “shell challenges” like you post above (it can do both the xargs challenge as well as the challenges linked in the blog post). If you’ve looked at a bunch of embedded shell libraries and formed an opinion about Rash by lumping it in with them without much inspection (not an irrational thing to do), Rash is much more powerful and flexible than you would assume. I looked at the related work and made real improvements. It’s not perfect, and there are mistakes I want to correct in future versions. And there are a few limitations of the platform I haven’t yet fixed (eg. I’ve started a patch to Racket necessary to allow Rash to include interactive job control, but I haven’t gotten around to finishing it. Maybe another improvement should be a general user-level safe fork, which I could use instead of Racket’s process spawning API to implement job control and be able to do all low-level Unix stuff with processes. This would lose the benefits of the portability of Racket’s process spawning API, though, and I like that Rash works on Windows.). But with time and work all these issues can be fixed. They aren’t intrinsic limitations to the idea of an embedded shell DSL.

                While you could come up with some challenges that do some specific low-level adjustment to how a process runs that Racket’s process execution API doesn’t cover, showing that Rash isn’t the best fit for certain low-level Unix-specific things, you could also come up with high level abstractions that are useful for live interaction and automation that low-level shells can’t do. Again, different visions of what the shell is and should be.


                As you argue, there is room for both our shells and more. I hope people can embrace nicer things than Posix shells for the various different visions of what “shell” means. My main hope in writing all this is that people don’t write off the idea of embedded shell DSLs because there have been so many failures and flawed implementations. There are very real and serious potential benefits of using embedded shell DSLs instead of stand-alone ones! There are also potential downsides, but the status-quo is generally that people use stand-alone shells when an embedded one could be better, not the reverse.

                1. 1

                  Thanks for your thoughtful comments. Short reply below, but on a different forum we could go deeper into this. Everyone is welcome to join https://oilshell.zulipchat.com and I’ve had conversations with other shell authors there.

                  I agree that a “narrow waist” shell that just supports the raw OS primitives should exist. However, I disagree that a large percentage of what I consider “shell scripting” should be done in that shell. Abstraction is powerful, convenient, and helpful.

                  This is a long argument, but I’d say I’m more concerned with composition than abstraction, because it makes programs shorter and more flexible. Shell functions are obviously a form of abstraction, but to me the key property is that they compose with external processes (e.g. the xargs example and the redirect example).

                  Pipelines are not particularly abstract, but they compose very well.

                  I would also say that certain forms of abstraction can inhibit composition.

                  The first two posts here are about composition more than abstraction I’d say, although it’s not either-or:

                  https://www.oilshell.org/blog/tags.html?tag=shell-the-good-parts#shell-the-good-parts

                  Also, a key point is that the old stuff has to compose with the new stuff. I think a lot of alternative shells suffer from a bunch of new concepts that don’t compose with the OS primitives and lead to longer programs with more edge cases.

                  The idea of using the lowest common denominator tool, or the purity of encapsulating specifically the “narrow waist” of Unix, are generally of little concern or even anti-values.

                  It has to be elaborated on the blog more, but the narrow waist is a practical architectural concept that reduces the amount of code in a system and makes code shorter. It’s desirable to aim for O(M + N) amounts of code vs. O(M * N) amounts. A real example of this is that there are R-Python bridges, and I’ve seen people try to write R-Java bridges, etc. There is a clear O(M * N) problem there that is addressed by simply using CSV (or Oil’s QTT).

                  If you’ve looked at a bunch of embedded shell libraries and formed an opinion about Rash by lumping it in with them without much inspection (not an irrational thing to do), Rash is much more powerful and flexible than you would assume.

                  Yes I’d like to read more about it. I did read the Rash paper a few years ago.

                  To me the key benefit of the embedded shell DSLs is for people who know that language. That is, JavaScript programmers might prefer zx, and Python users might prefer plumbum, etc. for obvious reasons. It’s hard to learn a new language.

                  So I think you are arguing that there are strong reasons to use Rash even if you DO NOT know Racket?


                  Other Notes

                  • Oil is a rich language with convenience, but it is kind of designed “around” the narrow waist, rather than trying to replace it with something else. As mentioned, JSON-like data structures are secondary to byte streams, and I believe that’s a feature and not a bug. I know pretty much everyone disagrees on first glance, so that’s why I’m devoting so much space to this on the blog.
                  • I do think the language runtimes are an issue as well; I remember reading that the Shill shell was originally written in Racket but they started to write their own intepreter when they wanted to productionize it. (I haven’t kept up with it though). One of my blog posts also links to a discussion about the Go runtime and fork().
                  1. 1

                    Re: O(M + N) vs O(M * N)

                    Yes, this is an important point. And if you are creating an embedded shell language within every general-purpose language, it is clearly an O(M * N) situation. Maybe only a few languages will end up with good embedded shells. But a shell embedded in any language can still be the glue language for programs written in any language.


                    Re: Narrow Waist

                    The Narrow Waist of Unix is too narrow. Lisp Machines had some much better ideas here, and PowerShell shows what some of that can be for a shell. Of course, .NET is not the narrow waist I want either, and PowerShell is basically useless outside of Windows. But it has some great ideas.


                    Re: composition vs abstraction

                    The key strength of embedded DSLs is that they compose better with their host language than they would as separate DSLs. I think embedded shells are a win for composition and abstraction.

                    There are limits to composition where you have rich data and maybe want to put it through a Unix pipe. But this is just dealing with the fact that the Unix waist is too narrow. So you have to hack around it by (potentially lossy) serialization, by passing functions by $0 reference or closure substitution, etc.

                    You mention that Oil is rich but designed around the narrow waist. I would say that Rash is designed to be a language that embraces a much wider waist, but that gracefully cooperates with the Unix waist where necessary. IE it’s designed so that one could write an ls in Racket that returns the listing as objects rather than strings, then the next pipeline stage could filter based on attributes of the objects like PowerShell, etc, but also has special consideration for Unix subprocesses and byte-stream pipes. While I can dream of a future with modern Lisp Machines, I live in a world with Unix. I designed Rash with both in mind.


                    So I think you are arguing that there are strong reasons to use Rash even if you DO NOT know Racket?

                    Yes, basically for 2 reasons: (1) because embedded shells have advantages, and Rash is, in my opinion, currently the best design of an embedded shell that can provide those advantages, and (2) because of strengths that arise specifically from using Racket. People who know Python and not Racket can pick up Xonsh more easily to get some of #1, but they will miss out on #2.

                    #2 is sort of a double reason, though. There is currently no other language that can host an embedded shell as well as Racket due to various language features that are unique to Racket (or relatively rare but found in Racket). These features are important to the implementation of an embedded shell that make the integration tighter and cleaner, as well as providing power and flexibility to live interactions and scripting.

                    (The most important feature is Racket’s macro system, which is truly world-class and far ahead of any competition.)

                    Ultimately encouraging someone to learn Rash implies encouraging them to learn at least a little bit of Racket, because you can’t really use Rash without some basic knowledge of Racket. Or in other words, if in Rash you want to use a data structure, or an if expression or loop, you have to use Racket because Rash just punts everything to Racket except a few features that are specific to shell programming. But I argue that features of Rash, features of Racket, and the synergy between them, provide things for shell programming that you can’t currently get elsewhere.

                    I do think the language runtimes are an issue as well.

                    Yes. While Racket is world-class in some ways, it also has weaknesses. The worst one for Rash is probably the slow startup time. Racket is reasonably fast once it’s running (eg. it’s faster than Python, which isn’t saying much. Scripts that I’ve ported from Bash to Rash end up being faster if they aren’t dominated by startup time.). But the startup is painful for scripts that are otherwise fast. I really want other languages to start taking key features like macros seriously so they can compete. Right now Racket really stands alone in its advantages.

                    I haven’t followed Shill that closely, though I read up on it several years ago. I know their first hurdle was that Shill required a custom kernel module that was only available for FreeBSD (I think). Shill is going in a very different direction than Rash. It is much more concerned with security and information flow, and much less concerned with other aspects of programming expressiveness or convenience. Shill issues with the runtime I think mostly revolve around the fact that they needed fine-grained control over (custom) OS security stuff while not benefitting very much from the expressiveness of embedding in a host language (with respect to their core goals).

                    That said, the runtime can also be helpful. If you are writing a shell language from scratch you have to write all the runtime stuff yourself. With Rash I got to just lean on Racket. Thus, Rash’s implementation is pretty small, just a few thousand lines (about half of that is the core library for pipelining and stuff, which can be used by itself like a Python user would use Plumbum, and half about macro stuff and parsing for the interaction-friendly syntax). If I write the line editor that I want to, I’m confident it will be much bigger than the entire DSL implementation.


                    It’s hard to learn a new language.

                    And whatever shell you learn is… another language to learn. An embedded shell gets to piggy-back. If you already know the host language, there is less to learn. If you don’t already know the host language, you need to learn some of it, but then you are learning a shell and a general purpose language for the price of one (or maybe 1.5).

                    I think embedded shells have a good story here for education. I think we both agree, and I think this phrase that I often use is actually a near-quote from you or another alternative shell author, that “it’s hard to tell someone to learn Bash with a straight face”. The shell is so useful, and unlocks so much practical power for people to harness. But not only is it yet another thing to learn, but Posix shells are particularly bonkers, full of weird gotchas for a learner (and even seasoned users) to trip over! This is a real problem – I see plenty of people graduate with a BS in CS, but with no ability to use the shell because it’s a weird thing that is pretty big and doesn’t really fit well in the school’s curriculum. These students have been kept in the dark about one of computing’s most empowering secrets that lets people, with relatively minimal work, write programs that actually do things they care about! (I could go on about my love for the shell and how empowering it was to learn it. I’m probably preaching to the choir here.) Now, a well designed modern shell of any type could improve the situation. But an embedded shell in a language the students already learn? It could help a lot.

                    Now, Rash is pretty complicated, to be honest. It has a lot of features that are there because they make it more powerful, but aren’t exactly pedagogical home runs. One thing I like about Racket is its simplified student languages. If you aren’t familiar with them, Racket has a series of simplified languages that remove various features. This allows the student languages to have better error messages and make it easier to learn both the syntax and semantics of the language in gradual steps. IE you can’t accidentally use some syntax for a more advanced feature and therefore get some error message that you can’t understand. I hope to some day make a “Student Rash” language that is less flexible, but provides an easy way for a student to learn the key concepts of shell programming as a “diff” to the programming they already know. Students could learn about processes, argv, pipelining, etc, but not need to learn yet another form of conditional, loop construct, syntax for the data structures they know, etc. They can keep using all of the things they know, in exactly the same syntax, and just learn the new concepts and syntax for running pipelines.


                    but I also want to encourage you to write a blog!

                    I do have a blog, but sometimes I feel more motivated to write when its in response to some specific question or comment. I have a lot of blog posts (about Rash and other things) that I’ve been meaning to write (along with a revamp of Rash’s documentation), but… I guess I’m more prone to putting off writing when there isn’t a specific person I’m responding to right now. Anyway, I do intend to write a bunch more about Rash and shell thoughts in my blog and in Rash’s documentation and introductory material. At any rate, I’ll file away my comments here as a starting point for writing future blog posts that explore the ideas more thoroughly (and hopefully more clearly).

                    That said, besides bombing lobsters threads about shells and using it to write all of my shell scripts, Rash doesn’t have a lot of my attention for the moment. I’m trying to wrap up my dissertation and find a job, and I’m not sure when I’ll have much time to focus on shells again. Hopefully soon. We’ll see. When I do I’ll definitely take another good look at Oil and other shells for good ideas.

                    Also, thanks again for maintaining such a great catalogue of shells on your wiki. It’s such a great reference to keep up on the shell world and what other people are doing.

                  2. 1

                    Sorry, my posts always end up really long when I’m talking about shell stuff. I should have edited that to be shorter. But I have a lot to say about shells…

                    1. 1

                      No problem, I will respond to this, but I also want to encourage you to write a blog! I would like to see how others are approaching their designs.

                      In particular I would like to know how a Racket-based embedded shell language compares with a Clojure, Common Lisp, or Elisp one.

                      And how those compare with Python/JS and Haskell/Ocaml ones etc.

                      I’m sure there is a lot we can learn from each other – I have read almost everything I can about shell, but there is still a lot missing IMO

                  3. 1

                    Somehow I missed that oil has json-like structures as a feature. With json being used more and more to glue languages together, I would imagine this fits well with your notion of shell as a glue language.

                    Certainly shell is used this way, but is that its purpose? It seems to me like shell and kernel are related words, the shell being a low level user interface on the outside, and the kernel on the inside. If shell is to be simple and powerful, then perhaps usefulness as a glue language emerges as a side effect.

                    1. 1

                      Well shell has many purposes and people have many viewpoints on it, as I mentioned in some long comments on this thread.

                      But while something like Jupyter notebooks could replace some use cases for a shell (not for me but for many), I’d say that the glue / narrow waist / least common denominator role is unique to shell.

                      Evidence I retweeted today :)

                      https://twitter.com/ozaed/status/1428421032096960516

            2. 4

              Why would anyone use this? I mean, I’d much rather use Perl than javascript on my servers.

              1. 20

                Because a great many people prefer JavaScript to Perl, or at least know JavaScript better than Perl.

                1. 24

                  Not everyone is you. Hope this helps.

                  1. 3

                    A use case would be using Pulumi for provisioning and zx for configuration, deployment and automation, with common libraries between both.

                    1. 2

                      I gotta admit JS leaves me wanting when it comes to really basic stuff like list comprehensions and dictionary syntax, but being able to throw together an async-y tool to run stuff in parallel cleanly, for example, is a pretty great party trick.t

                      For example, you can easily make your build scripts just follow their dependencies cleanly without having to go full Bazel to get the proper speed boosts you might want