1. 14

  2. 7

    Using pickle for data marshaling in an intrusion detection project? No thanks, Lockmart.


    1. 4

      My pet peeve these days is:


      No, damnit! #! /usr/bin/env python

      1. 4

        I recognize the potential problem with the “direct” approach, but it seems like the env approach just trades one problem for another (namely, that a user with a $PATH that prefers a different python interpreter than the script’s author intended can end up with unexpected breakage – a situation I find myself in somewhat often, for various reasons). Personally, I very much prefer the obvious, unambiguous immediate-ENOENT failure mode of the former to the who-knows-what-might-go-wrong-or-how-or-when of the latter, so I stick with the plain old hard-coded absolute path.

        1. 3

          Chalk me up as another hater of non env shebangs in scripts. Don’t presume /usr/bin/python is what you want, all the problems you specify with PATH being messed up exist with /usr/bin/python too.

          What if /usr/bin/python is python3 and not python2? What if the user installed python to /usr/local/bin? What if their script is running on nixos where jack and squat is in /usr/bin?

          If a user modifies their PATH to point to a different interpreter, that is the whole point of the env shebang line. Unexpected breakage? Well yeah, but if we stick to /usr/bin/BLAH we get the same problem.

          With env? PATH=/python/2.7/bin:$PATH scriptname vs /python/2.7/bin/python $(which scriptname)

          Want to run it on 2.6? PATH=/python/2.6/bin:$PATH scriptname vs /python/2.6/bin/python $(which scriptname)

          Alternative without env means you specify python entirely or rewrite the shebang line to be the sane env line. Assuming /usr/bin/python exists on your target system is as bad of an assumption here. A worse one in my opinion. You’re probably OK if you only ever run on linux, but as someone that had to deal with multiple unixes in the past, env is by far the better iocaine powder here.

          1. 2

            Yes, I was a bit curious what is there on NixOS, so…

            $ ls /usr/bin
            1. 4

              Yep, nixos is the most unique unix like distro in this regard. But /usr/bin/env is present there for a good reason and for shebang lines is basically it. But if you’ve ever written “portable” scripts where some things are in /usr/sfw, other os’s in /opt, etc…. /usr/bin/env is a real nice thing to use and depend on.

              Besides I consider /usr/bin/… to normally be out of date, wrong, compiled wrong, doesn’t have the things I need, etc… But I’ve had to deal with too much old unix probably and am just angry at hpux and solaris at this point.

              1. 2

                It’s… ultimately, any script relies on the user to tell it where the tools it needs are. $PATH is a mechanism for that, and much simpler than editing the source, which is the major alternative. If the user doesn’t think about it or gets it wrong, bad things happen, and that’s not avoidable.

                Or we could avoid it by making all Unix software portable. hollow laugh

                1. 2

                  Or we could avoid it by making all Unix software portable. hollow laugh

                  Sadly so so much jaded truth in that sentence. “portable” scripts is one of the funniest or saddest things ever inflicted on the world. You should start a comedy routine! >.<

                  1. 1

                    Thanks. ;)

          2. 2

            That can work for stuff that you make for yourself, but when you try to distribute something you either test every OS/distro possible or.. you just use env.

            As a FreeBSD user I’m really sick of people going /bin/bash.. or even worse, doing /bin/sh assuming it’s an alias of bash.

            1. 1

              I guess I understand that. It does completely fail on NixOS, though…

            2. 1

              I like the setup tools console script entry point. pip install and it takes care of all the OS related stuff.

              1. 1

                This is why I like vagrant for managing OS deps and virtualenv for building python deps. You can share code that bootstraps itself with its env variable and executable path, installs the python libs, and then when the user runs it with something like a shell script all the path references are in place, and the only sudo usage required is to install pip and virtualenv (if not included in the distro already)

                1. 2

                  This is why I hope Nix (or the equivalent) becomes more popular, it does what virtuanenv does but at the level of the whole system.

              2. 2

                Based on the giant red warning message on python’s website, I’d say you’re right.


                1. 2

                  There is an important distinction to make here: that warning says not to unpickle a pickle file from an untrusted source, it’s not a warning about pickling malicious data. I don’t know if that’s a difference in kind or not, but I think the case can be made that the first is a 100% terrible idea and the second unwise but more difficult to exploit.

              3. 1

                This is really nice pragmatic approach to solving a difficult problem. A lot of stuff to learn from here.