PyLint - W0102, dangerous-default-value
PyLint - W0102, dangerous-default-value
Please use an IDE or lint your code to prevent this.
That doesn’t change the fact that’s is recurring WTF. Either you’ve worked with python for a while and have it internalized (or an IDE), but blaming an actual shortcoming of the language on the developer isn’t helpful.
To me most lint tools are overly aggressive, and good warnings like this one get drowned out by bad ones.
FWIW the google python style guide disallows mutable default arguments: https://google.github.io/styleguide/pyguide.html#212-default-argument-values
But I think it would better if there was some non-lint / non-IDE / non-google place where this knowledge is stored and linkable.
Googling reveals a book: https://docs.python-guide.org/writing/gotchas/
People still rediscover it not just in 2021, but 2018 too: https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/ (I’ve known about this issue approximately since I started using Python, which was ~2003 or so)
The first time I encountered this behavior was through a linter warning, which can be seen in my original comment. I did the research and I understood what was happening. Then I changed my code accordingly and never made that mistake again.
The only thing I „criticize“ is that the developer learned about this well known behavior of Python, because of a production bug. I wanted to point out that there is another way.
How should the default-value parameter be assigned when the function is called? I see four options:
They all have different drawbacks.
Naïvely, I would expect a default value to behave identically to having the expression written in at the call site.
I would expect
To behave identically to
Scoping issues aside (I would expect syms in expr to be bound at the definition site.)
So, 2. I think this is what most people expect, and why this decision is so surprising.
Are there languages that do (3) and (4)? And any language other than Python that does (1)?
Python is the only language I can think of right now where this is a caveat. Maybe C. But even there you deliberately hand in a reference if you want that to be mutable.
PyLint is overly aggressive, and doesn’t catch all instances. For e.g. see.
self.val = 
def process(self, e):
def processed(element, o = MyC()):
Honestly, are there any Python tutorials that explain default values that don’t tell you not to do this? It sucks and it’s a real flaw in Python that this can happen, but to me, this is an extremely junior mistake to make. Unless you’re Digg and there are no senior Python engineers because Python is a new language, someone should have caught this in code review.
This sort of mistake is what someone (if not most) coming from other languages with default values would do, not so much juniors reading tutorials, I think.
The more traps there are in a language, the less safe it is. I wouldn’t disagree that this is a junior mistake – you only need to make it once, but on the scale of language safety, what matters is how many bugs it produces. Some languages are for experts, and are unsafe – C, whereas other languages are made to be simple – shellscript, and yet manage to screw it up with quoting rules that attract bugs like fly paper – very much a junior bug too. So I would absolutely say that unsafe languages exist in both ends of the simplicity spectrum and that this qualifies.
(This was submitted at hn.)
The problem is
def append_to(element, to=):
I default to using a @fix_params (below) now because I like having default arguments for my functions.
def wrapper(*args, **kwargs):
original_defaults = func.__defaults__
func.__defaults__ = tuple([val() if callable(val) and
else val for val in func.__defaults__])
rval = func(*args, **kwargs)
func.__defaults__ = original_defaults
I use a lambda with empty arguments to indicate where I want a newly initialized default value. With this, one can do:
def append_to(element, to=lambda: ):
def processed(element, obj=lambda: MyObj()):
And expect it to work.
This does mean that one loses the ability to provide empty argument lambdas as default arguments in the annotated functions.
Here I would suggest a different fix. Change the default value to an empty tuple, then create your own list. This way you aren’t mutating the callers argument and get an error if you forget to copy the argument. Basically I see this change as a partial fix that handled the default, but not passed argument case.
Mutable values are a huge liability and need to be handled with care. By default you should clone all of your arguments and return values so that surprise mutations don’t cause bugs. Of course this has a performance cost in most cases so there are always going to be lots of exceptions.
When there are a lot of parameters to pass and there are some defaults, I use dataclasses. You get a field(default_factory=list) syntax and you are able to pass the parameters as a group to other functions as well.