1. 31
  1.  

  2. 8

    My (small) contribution was directly caused by a comment on lobste.rs that Oil Shell should have a man page. I then got the chance to learn about writing man pages/mdoc and did a little more investigation to autotools/automake.

    I’d like to thank @andyc for doing an outstanding job of making contributions easy and feel welcome.

    1. 3

      Yes thanks for the help! That was something multiple people pointed out.

      As mentioned in the post, we still need to work on the quick ref and help builtin!

    2. 4

      I have an idea for a blog post: How to Rewrite Oil in C++, Rust, or D. If you’re interested in that, leave a comment.

      that sounds extremely interesting! a python/c -> d conversion in particular would be fantastic because d seems like it would naturally accommodate straightforward translation of code from both languages, and possibly even incremental migration.

      1. 5

        Yeah I found D interesting because it has garbage collected data structures, unlike C++ or Rust. That makes it the most natural target of translation for a typed Python codebase, which doesn’t have any notion of deallocation.

        (Although I’m personally working on C++, because I’m familiar with it, build dependencies matter for a shell, and because shell depends a lot on libc, and libc bindings are trivial in C++.)

        But the basic idea is that I wanted to encourage other people to try to rewrite Oil in a different language, in case I fail :)

        I believe in the power of parallel development in open source. Since there are at least 5 major, independent JS engines, there could probably be 1 or 2 more shell implementations.

        You would get a “leg up” in the following sense:

        • The Oil lexer is specified with regexes, which can be ported to any language easily. It does a huge amount of work, so you don’t have to “grovel through backslashes and braces one-by-one”, as I like to characterize the implementation strategy of other shells.
        • The Oil ADSL schemas, e.g. syntax.asdl. They are also code generators that are easy to port, and represent a huge amount of “logic”.
        • The parsing algorithms documented on the blog

        So it would be significantly less work than writing a shell from scratch.

        I have a few blog posts in front of that, but if people are interested, let me know and that will motivate me to write it.


        Random: I’ve also had the idle thought that since Fabrice Bellard just released a huge JS interpreter written from scratch, and has done a C compiler, maybe he will want do also do a shell :) The goal of OSH was really to “discover a spec” and I think that has already succeeded.

        That said, I’m also going to try to do it myself. But it is a huge amount of work, and as mentioned I’m also working on the Oil language, which is significantly larger than OSH. And everything needs documentation, etc.

        1. 5

          Do consider Nim as well. Probably a better alternative to D in this case, since it also has Pythonic syntax, which might aid in the translation process.

          1. 3

            I honestly don’t think that matters very much. Aside from whitespace for indentation, the syntax isn’t especially more pythonic, and switching out indentation for braces is far from the most interesting piece of the puzzle.

            Of course, I might be a bit biased, since I prefer d to nim :P

            1. 3

              I will try to make the title more language-agnostic. The idea is to encourage with people with expertise in a particular language to think about how they would do it, and maybe try it. The content of the post itself is mostly language agnostic.

            2. 2

              I’m definitely interested. I enjoy rewriting code, both refactoring and porting it to different languages, and it would be neat to join in an effort to do that for oil.

              I probably can’t do it singlehanded due to the sheer amount of work involved, but it would be a fun project to chip away at, and I bet a blog post with a roadmap of sorts would spark tons of interest.

              1. 3

                OK awesome! Yeah I enjoy rewriting code too, especially when there are tests, which Oil has a lot of. I liked adding all the type annotations, which feels like a form of refactoring. Only about half of it is type annotated, and I’m looking forward to annotating the rest.

                I will bump that up on the list… First I have to publish the draft of The Interactive Shell needs a Principled Parser, and then I’ll try to do this one.

                It’s partly “defensive” in that I get a lot of uninformed “Why Don’t you rewrite it in X?” comments :) But it’s also 100% serious – I do believe that the diversity of ideas in open source moves things forward faster. That includes diversity in languages.

              2. 1

                Since Oil is already type annotated, why not just use mypyc to compile to native code?

                https://github.com/mypyc/mypyc

                1. 2

                  Yeah good question. Oil isn’t fully annotated yet (only about half of it is), but I may try it.

                  The main issue is that I want to completely remove the dependence on the Python interpreter, while mypyc generates Python extensions. So after mypyc you still need a Python interpreter.

                  In March I visited Guido van Rossum at Dropbox, and we talked about MyPy and mypyc (and Oil). This “mycpp” experiment is heavily inspired by mypyc and Shed Skin:

                  https://github.com/oilshell/oil/tree/master/mycpp

                  I have a bunch of test cases working, but there are still a few more things to get working before I could apply it to Oil.

                  1. 1

                    Dropping the Python interpreter requirement makes sense. It’s a lot of code. I wonder how easy it would be to build a stripped-down version of the Python interpreter for “embedded” use cases (no parser / compiler / interpreter, just enough runtime to run Python extensions). Going further, I wonder how easy it would be to statically bundle all the required extensions. I imagine many mypyc-based projects would benefit from a mode like this.

                    1. 1

                      Yes, I did that for Oil. The whole thing is statically linked and it doesn’t include Python’s parser or compiler.

                      http://www.oilshell.org/blog/2017/05/05.html

                      http://www.oilshell.org/blog/2018/11/15.html

                      It requires a lot of hacking and it’s difficult to generalize, due to the way CPython is written. (Though there is a Rust project trying to do it, I think PyOxidizer). You really need some cooperation from upstream, and upstream there have been many different half-measures over the years.

                      But yeah the next step is to go even further and get rid of the whole thing!

                      1. 1

                        I don’t think you removed the interpreter too, but that would require adding type annotations to all standard library code on which you rely. In any case it looks like you’ve thoroughly explored this route!

            3. 1

              I think I had a similar question last time around, but: Should I be using 0.6.0 or 0.7.pre1?

              1. 2

                @andyc can correct me, but I think the default answer is: the later the release, the better.

                1. 2

                  Yes @msingle has it right – later is better in almost every way! Every release undergoes the same big suite of automated tests, and the numbers are constantly getting better. Results are towards the end of the release page:

                  https://www.oilshell.org/release/0.7.pre1/

                  The only exception is that I manually tested 0.6.0 on OS X but I didn’t on 0.7.pre1.

                  But 0.7.pre1 happened to have some other portability fixes as mentioned in the release notes.