1. 11

  2. 5

    I plan to work on these things […] A “carrot” for Oil, i.e. a feature that existing shells don’t have. Ideas: [static analysis, app bundles, and crash reports]

    If you’re looking for ideas, here’s another one that might interest you: nestable string literals, a.k.a. tagged string delimiters, a.k.a. I don’t think this construct has an official name. It’s a rare language feature AFAICT, but solves an annoying and error-prone task that is especially common in shell: quoting, escaping, and multiple escaping.

    Examples of tagged string delimiters in other languages (I am aware of only two):

    • Lua’s long brackets let you write string literals (and multiline comments) as [[...]], [=[...]=], [==[…]==]`, and so forth – the number of equals signs in the opening delimiter is counted, and the string ends when the matching closing delimiter is found.

        --[[ This comments out an assignment to my_lua_string
        my_lua_string = [==[one [=[inner]=] two]==]
        -- This is a string delimited with long brackets, that contains several other closing delimiters that are ignored without needing escaping
        [=[one ]==]'" two]=]
        --> 'one ]==]'" two'
        -- Using long brackets with loadstring (Lua's `eval`):
        f = loadstring([[i = i + "x"]])
        i = 'a'
        f()  -- i = 'ix'
        f()  -- i = 'ixx'
    • PostgreSQL’s dollar quoting:

        $$Dianne's horse$$
        $SomeTag$Dianne's horse$SomeTag$
            FUNCTION increment(i integer) RETURNS integer AS $myAddOne$
                    RETURN i + 1;
            $myAddOne$ LANGUAGE plpgsql;
    • Heredocs don’t count, because you can’t write them inline.

    Ways tagged string delimiters would improve the OSH/shell experience:

    • Tagged string delimiters make it trivial to write any string literal without escaping the contents: you simply choose a delimiter that doesn’t occur in the string.
    • Tagged delimiters make it easy to nest commands with quoted string arguments inside the quoted string argument of a higher-level command: for example, cp 'my file with.spaces' /etc within sudo sh -c 'multiple; commands; cp "..."' within ssh 'multiple; commands; sudo sh -c "..."'.
    • Shell’s string-centric nature and interactive usage means is the context in which I frequently quote strings, or pass quoted shell code that contains its own quotes that need escaping, and so on. The example in the previous item is one I have encountered in real life, albeit only once. It would be good to get away from escaping, and wonderful to get away from double escaping.

    Sorry to dump such a long post on you (although turnabout is fair play :-P), but I thought you might be interested. For my part, I find your OSH project fascinating, and avidly read every post. Thank you for writing such good writeups!

    1. 5

      Perl has nestable strings with q{} / qq{} / qx{} as well.

      print q{foo q{bar}};
      1. 3

        Yes I totally agree that shell’s string-centric nature means that this is one of the most important features! I often have HTML snippets in my shell scripts, and I see tons of config file snippets in shell scripts, like /etc/resolv.conf, etc. Not to mention Awk, Perl, and Python snippets.

        I think of this feature as “multiline strings”. It will subsume here docs, and as you mention you can use one as an argument to a command.

        It will replace all the variants of here docs: << EOF , << 'EOF' (quoted), <<-EOF (indented), etc.

        I’m thinking using a variant on Python’s syntax:

        # operator << takes a string literal, not a filename, like <<< in shell
        cat << 'normal string' 
        cat << '''
        $var is NOT expanded in 3 single quotes, equivalent to quoting 'EOF'
        cat << """
        $var is expanded in 3 double quotes, like unquoted EOF

        I have thought about the “tag” problem too. I originally had a proposal to have some sort of tag, but someone pointed out to me that it’s not necessary for multiple here docs on a line. You can just use the order of the here docs.

        I guess I have never really had an issue with Python’s multiline strings – e.g. embedding a multiline string in a multiline string! The fact that there are two different types of quotes helps. But I’m open to adding that later if it’s a problem.

        I plan to write a preview of Oil syntax, as requested on lobste.rs. So I’ll make sure to highlight this part. Unfortunately I have at least 2-3 posts to write before that, so it might not come for awhile.

        There is also the issue of C escapes, e.g. $'\n' in shell.

        Thanks for the feedback!

        1. 1

          multiline strings with interpolation and also trimming of leading indentation is super handy, the way nix does it is pretty fun to use.

          1. 1

            OK interesting, I didn’t know Nix did that. That’s pretty much what I expect Oil to have – the indentation of the closing quote is what you strip off of every line.

            But ‘’ already means something in both shell and Oil, so it will be ‘’’ and “”” like Python, except that single and double mean what they already do in shell.