1. 20

  2. 7

    Only in the Caveats section does it hint as to what OSH is (“Think of OSH as a programming language rather than an interactive shell. It has basic interactive features, and I want it to be a good bash or zsh replacement, but it’s not there yet”).

    1. 2

      If you click through on the OSH text, it gives you more of a description:

      A statically-parseable language based on the common use of shell, in particular bash. In almost all cases, it’s indistinguishable from bash.

      Also clicking on the upper right “Home” gives more info. Happy to answer any questions though!

      1. 5

        He means that the explanation of OSH should be given first. A user shouldn’t have to navigate around to get that crucial information.

        1. 3

          OK fair enough, I edited the post. I’ve had many posts on Lobsters over the last several months, so I thought people would know, but it’s good to have it in a release announcement.

    2. 4

      I downloaded, compiled, and ran oil on Debian Testing, and it worked fine, but that’s no big surprise.

      In terms of test-scripts, the shell-based build system redo includes a test suite it uses to find the most appropriate shell to run build-scripts with; in its own words:

      Note that this file isn’t really a test for POSIX compliance. It’s a test for usefulness-compliance, that is, interpreting POSIX in a particular way for consistency, so that users of redo can depend on all the following functionality.

      Currently osh fails or produces warnings for 18 of the 116 tests, so that might be help you prioritise which functionality to add next.

      Also, in other contents I notice you express a preference for shell-based build scripts over shell-in-Makefile build scripts; perhaps you could consider using redo? If the end-user has redo installed, they get all the usual benefits of Makefiles (parallel building, incremental building). If the end-user doesn’t have redo installed, there’s a small shell-script you can ship with your software that just runs the build without all the incremental smarts.

      1. 1

        I’ve seen redo, but I hadn’t seen the test suite. I will check it out – thanks for the pointer!

        I’m a fan of most things DJB, and redo is nice. But I think it’s too minimal – the lack of any real projects using it is probably a signal. I probably won’t use it directly, but I may take inspiration from it.

        Off the top of my head, redo causes you create lots of tiny little files. I think it’s best for small things. And I also don’t know how I would cleanly support build variants in redo (debug, release, asan, etc.)

        I wrote about it a bit here:


        As well as another shell-based build system I found, blur.

        I think my main beef is the syntactic conflict between Make and Shell. It’s just ugly. So many essential characters collide, like \ and $ and more.

      2. 3

        As a Pythonista I am quite fascinated that you actually rewrite the Python build system - definitely a very interesting project (not just because of that) :)

        I just tried it on Arch Linux:

        make runs through but is moaning a lot (see https://pastebin.com/qN8KNxdU)

        Trying to run it causes an assertion error:

        18:00:51 oliver@ob1 [134] < ~/incoming/oil-0.0.0 >  8963 %
        osh -c 'echo hi'
        osh: Modules/main.c:347: Ovm_Main: Assertion `sts != -1' failed.
        zsh: abort (core dumped)  osh -c 'echo hi'
        1. 3

          Oh that is bad, let me try to reproduce it.

          Do you know if Arch has a chroot image? One thing I like about Alpine Linux is that they provide their root file system [1] so I can just “chroot” rather than using virtualization. That uncovered a lot of bugs, because the only thing that matters is really the C environment.

          I did a little Googling and it says you need the “devtools” package for a chroot, but I don’t have pacman on my Ubuntu system obviously.


          If not, it’s not a big deal. I can probably just use QEMU.

          Thanks for trying it!

          [1] https://alpinelinux.org/downloads/

          1. 3

            Could also do something like docker run -it dock0/arch /bin/sh

            1. 1

              I have not much experience using chroot other than using it to install Arch. For testing on other systems I like to I use vagrant as it maps your project into the machine automatically. Here is an arch box for example: https://app.vagrantup.com/terrywang/boxes/archlinux

            2. 2

              Thanks, I filed a bug here. I will probably set up some QEMU images for this:


            3. 2

              You abstractions around the build system make it hard to track down build issues:

              build/compile.sh build-opt _build/oil/ovm _build/oil/module_init.c _build/oil/main_name.c _build/oil/c-module-srcs.txt
              /usr/local/build/oil-0.0.0/oil-0.0.0/Python-2.7.13 /usr/local/build/oil-0.0.0/oil-0.0.0
              Objects/fileobject.c: In function 'file_write':
              Objects/fileobject.c:1805: warning: 'err' may be used uninitialized in this function
              Objects/unicodeobject.c: In function 'PyUnicode_DecodeUTF7Stateful':
              Objects/unicodeobject.c:1694: warning: comparison is always true due to limited range of data type
              Modules/posixmodule.c: In function '_PyInt_FromUid':
              Modules/posixmodule.c:358: warning: comparison is always true due to limited range of data type
              Modules/posixmodule.c: In function '_PyInt_FromGid':
              Modules/posixmodule.c:366: warning: comparison is always true due to limited range of data type
              1. 5

                Yeah it’s a little weird, and there is a bit of ideology in the code [1]. I prefer to use shell rather than shell-in-make. So the Makefile calls out to build/*.sh for anything complicated.

                And it’s more weird because I’m embedding the whole Python interpreter, but I abandoned its autoconf scripts. I think that non-Linux users will appreciate this eventually, since the configure script runs fast and doesn’t do ridiculous things [2]. But it will take some more work to get it correct.

                So the problem is as mentioned here:


                I froze pyconfig.h based on a ./configure run from Alpine Linux with gcc and musl libc. But obviously that will produce some incorrect results for OpenBSD. Python runs fine on OpenBSD AFAIK so those problems should be pretty easily fixable.

                Although some of those warnings might be in totally unused code. posixmodule.c is big, and I’m only using a portion of it in OSH. The C compiler can’t get rid of that code because it doesn’t know if it’s dynamically called from Python – function pointers are embedded in tables.

                So my reply is the same: if you have suggestions about where to get (non-root) SSH access to an OpenBSD environment with vim/git/bash (and hopefully tmux), that will help me fix build issues. I appreciate patches too, but I feel like they will break without a test environment.

                The theory is that a shell shouldn’t require a lot of #ifdefs generated by a big configure script – it should just rely on POSIX and ANSI C. I might regret abandoning autoconf but it seems like a good way to learn how C and POSIX platforms differ in practice.

                [1] http://www.oilshell.org/blog/2016/11/13.html

                [2] http://queue.acm.org/detail.cfm?id=2349257