1. 17
  1.  

  2. 18

    PyLint - W0102, dangerous-default-value

    Please use an IDE or lint your code to prevent this.

    1. 10

      That doesn’t change the fact that’s is recurring WTF. Either you’ve worked with python for a while and have it internalized (or an IDE), but blaming an actual shortcoming of the language on the developer isn’t helpful.

      1. 7

        To me most lint tools are overly aggressive, and good warnings like this one get drowned out by bad ones.

        FWIW the google python style guide disallows mutable default arguments: https://google.github.io/styleguide/pyguide.html#212-default-argument-values

        But I think it would better if there was some non-lint / non-IDE / non-google place where this knowledge is stored and linkable.

        Googling reveals a book: https://docs.python-guide.org/writing/gotchas/

        People still rediscover it not just in 2021, but 2018 too: https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/ (I’ve known about this issue approximately since I started using Python, which was ~2003 or so)

        1. 7

          The first time I encountered this behavior was through a linter warning, which can be seen in my original comment. I did the research and I understood what was happening. Then I changed my code accordingly and never made that mistake again.

          The only thing I „criticize“ is that the developer learned about this well known behavior of Python, because of a production bug. I wanted to point out that there is another way.

          1. 3

            How should the default-value parameter be assigned when the function is called? I see four options:

            • it’s assigned the object obtained by evaluating the default-value expression when the function was initially defined (the current way,
            • it’s assigned the object obtained by freshly evaluating the default-value expression, on every call,
            • it’s assigned a deep copy of the object obtained when the function was initially defined,
            • only known-immutable python values are allowed as default values.

            They all have different drawbacks.

            1. 4

              Naïvely, I would expect a default value to behave identically to having the expression written in at the call site.

              With:

              def f(a=expr)
              

              I would expect

              f()
              

              To behave identically to

              f(expr)
              

              Scoping issues aside (I would expect syms in expr to be bound at the definition site.)

              So, 2. I think this is what most people expect, and why this decision is so surprising.

              1. 2

                Are there languages that do (3) and (4)? And any language other than Python that does (1)?

                1. 2

                  Python is the only language I can think of right now where this is a caveat. Maybe C. But even there you deliberately hand in a reference if you want that to be mutable.

              2. 2

                PyLint is overly aggressive, and doesn’t catch all instances. For e.g. see.

                class MyC:
                    def __init__(self):
                        self.val = []
                
                    def process(self, e):
                        return self.val.append(e)
                
                def processed(element, o = MyC()):
                    o.process(element)
                    return o.val
                
                print(processed(42))
                print(processed(51))
                
              3. 8

                Honestly, are there any Python tutorials that explain default values that don’t tell you not to do this? It sucks and it’s a real flaw in Python that this can happen, but to me, this is an extremely junior mistake to make. Unless you’re Digg and there are no senior Python engineers because Python is a new language, someone should have caught this in code review.

                1. 9

                  This sort of mistake is what someone (if not most) coming from other languages with default values would do, not so much juniors reading tutorials, I think.

                  The more traps there are in a language, the less safe it is. I wouldn’t disagree that this is a junior mistake – you only need to make it once, but on the scale of language safety, what matters is how many bugs it produces. Some languages are for experts, and are unsafe – C, whereas other languages are made to be simple – shellscript, and yet manage to screw it up with quoting rules that attract bugs like fly paper – very much a junior bug too. So I would absolutely say that unsafe languages exist in both ends of the simplicity spectrum and that this qualifies.

                2. 5

                  (This was submitted at hn.)

                  The problem is

                  def append_to(element, to=[]):
                      to.append(element)
                      return to
                  
                  >>> print(append_to(42))
                  [42]
                  >>> print(append_to('51'))
                  [42, 51]
                  

                  I default to using a @fix_params (below) now because I like having default arguments for my functions.

                  import functools
                  import inspect
                  
                  def fix_params(func):
                      @functools.wraps(func)
                      def wrapper(*args, **kwargs):
                          original_defaults = func.__defaults__
                          func.__defaults__ = tuple([val() if callable(val) and 
                                                       not inspect.signature(val).parameters
                                                       else val for val in func.__defaults__])
                          rval = func(*args, **kwargs)
                          func.__defaults__ = original_defaults
                          return rval
                  
                      return wrapper
                  

                  I use a lambda with empty arguments to indicate where I want a newly initialized default value. With this, one can do:

                  @fix_params
                  def append_to(element, to=lambda: []):
                      to.append(element)
                      return to
                  

                  or even

                  @fix_params
                  def processed(element, obj=lambda: MyObj()):
                      obj.process(element)
                      return obj
                  

                  And expect it to work.

                  >>> print(append_to(42))
                  [42]
                  >>> print(append_to('51'))
                  [51]
                  

                  This does mean that one loses the ability to provide empty argument lambdas as default arguments in the annotated functions.

                  1. 4

                    Here I would suggest a different fix. Change the default value to an empty tuple, then create your own list. This way you aren’t mutating the callers argument and get an error if you forget to copy the argument. Basically I see this change as a partial fix that handled the default, but not passed argument case.

                    Mutable values are a huge liability and need to be handled with care. By default you should clone all of your arguments and return values so that surprise mutations don’t cause bugs. Of course this has a performance cost in most cases so there are always going to be lots of exceptions.

                    1. 2

                      When there are a lot of parameters to pass and there are some defaults, I use dataclasses. You get a field(default_factory=list) syntax and you are able to pass the parameters as a group to other functions as well.