This is a bad article in my opinion, it doesn’t make any arguments, just assumes that you need a very good reason to do “operator overloading”, and since he can only think of one (a DSL) then that’s the end of the argument.
I reject the premise that operator overloading is inherently something to be avoided at all costs, in fact I think it’s very pythonic. sqlalchemy overloads operators on column to great effect, you could say it’s a DSL but I don’t know a clear dividing line between any library that becomes very well fitted to it’s domain and a DSL, in a dynamic language.
Further reject the premise that call is best thought of as operator overloading (or the sort people tell you to avoid). Some of the arguments for why to avoid overloading don’t apply - if you overload + for a class, then every time someone reads + in your codebase they need to be aware that the operands might include your class (where before it was only a small set of types allowed). There’s also two types involved so the dispatch can be complicated.
If you overload call, now they need to be aware that when something is called eg x(), it could be a function or a class. But that’s not really important since functions can take and return anything and store state, and so can classes. From a consuming perspective there really isn’t an important difference. There’s only one type involved so dispatch is much simpler.
His readability argument (he just states “more readable”), I can only think of 2 reasons, 1: you don’t know call, in which case go read a textbook. 2: call means you get 1 less name vs a named method. In lots of cases though, the name of the method would reasonably be identical to that of the class, these are the most common cases where I think call is appropriate.
You’re right in that it’s fine to use call when it is appropriate. However, I agree with the article that it’s not something you should be doing “just because you can”. If you look at code that instantiates an object and then uses the object as a function, it is more confusing than simply calling a method on the object. It is overly “cute” and non-obvious. In most cases, simple code that does not obscure what is going on is better than code that is slightly harder to follow.
Yes, as an experienced Python developer, you should be aware that an object may be callable via magic methods, but it’s way easier (especially for developers less experienced with the language) to follow code that doesn’t rely on such “tricks”. I’m a big proponent of using language constructs well, but I think their use is similar to “big” words in a natural language; you can eloquently express yourself by using them, but if you over-use them, it makes you seem pompous and harder to understand, even for people that have a large vocabulary. No everyone is at the same level you are. Sure, you can look it up in a textbook, just like you can look up new words in a dictionary. But not requiring the less fluent reader to do so unnecessarily is a mark of skill for an author.
Of course there are many situations where using callable objects is suitable: like you said the SQLAlchemy DSL, or situations where code requires a callable, but you need to do more bookkeeping than is practical with a closure (which would happen when you’re using an API in a slightly different way than intended). Right tool for the job and all that…
Yes, as an experienced Python developer, you should be aware that an object may be callable via magic methods, but it’s way easier (especially for developers less experienced with the language) to follow code that doesn’t rely on such “tricks”.
In my opinion code shouldn’t be written poorly so that it’s easier for newbies to follow. If you can make it easier for newbies to follow without sacrificing anything then that’s good, but in this exact case in the article, a method ‘get_name’ is used instead and that name is more confusing than just using __call__!
In my opinion code shouldn’t be written poorly so that it’s easier for newbies to follow.
I didn’t mean to imply that you actively dumb down your code. I just meant that using more “complex” features should be carefully considered and only used when necessary or definitely clearer.
If you can make it easier for newbies to follow without sacrificing anything then that’s good, but in this exact case in the article, a method ‘get_name’ is used instead and that name is more confusing than just using call!
Maybe a better name of the method would be store_matching_filename or something like that, because that’s what it is doing. You could also argue that this is a DSL that matches files so it’s fine to use __call__, and if it’s part of a bigger library or system with more such classes that might definitely be the way to go. But for a one-off file matcher I’m not sure I would do this. Of course, it’s quite a contrived example so it’s neither here nor there.
Independent of whether to make objects callable for this use case: get_name() was a puzzling choice for this particular method. Accessor methods often take this kind of name and this is not an accessor. If I needed an accessor with Python, I’d probably just access the attribute or create a @property-decorated method. But if I was reading code that called some get_name() I’d expect it not to mutate and I’d also expect it to return a value.
A use case that I like for __call__ is making a class that can be used as both a context manager and a decorator. Pretty handy for things like timing or handling specific errors in a consistent way.
I’ve overridden __call (in Lua, not Python, but it applies) but it’s not often I do. In one case, it was a wrapper for syslog(). The module includes three methods, open(), log()’ and close(). I really did not like the idea of having to call syslog.log() all the time, so I made the syslog object callable.
A callback/closure is more “lightweight” than a full-blown named class. You would also need to instantiate the class, which is not necessary with a closure: you just write it on the spot and it has access to the local variables. In Python, however, closures are a bit confusing or unusual, so I would argue that it might not be the clearest solution even here, especially since it’s arguing about simpler code :)
They’re equivalent, as a Schemer I know that in my bones. However, it certainly feels like a lot more ceremony is involved in Python at least to make an object: you must declare the class and methods on it, then instantiate it, and finally call the method. Defining a closure inline and calling it is more off-the-cuff.
This is a bad article in my opinion, it doesn’t make any arguments, just assumes that you need a very good reason to do “operator overloading”, and since he can only think of one (a DSL) then that’s the end of the argument.
I reject the premise that operator overloading is inherently something to be avoided at all costs, in fact I think it’s very pythonic. sqlalchemy overloads operators on column to great effect, you could say it’s a DSL but I don’t know a clear dividing line between any library that becomes very well fitted to it’s domain and a DSL, in a dynamic language.
Further reject the premise that call is best thought of as operator overloading (or the sort people tell you to avoid). Some of the arguments for why to avoid overloading don’t apply - if you overload + for a class, then every time someone reads + in your codebase they need to be aware that the operands might include your class (where before it was only a small set of types allowed). There’s also two types involved so the dispatch can be complicated.
If you overload call, now they need to be aware that when something is called eg x(), it could be a function or a class. But that’s not really important since functions can take and return anything and store state, and so can classes. From a consuming perspective there really isn’t an important difference. There’s only one type involved so dispatch is much simpler.
His readability argument (he just states “more readable”), I can only think of 2 reasons, 1: you don’t know call, in which case go read a textbook. 2: call means you get 1 less name vs a named method. In lots of cases though, the name of the method would reasonably be identical to that of the class, these are the most common cases where I think call is appropriate.
You’re right in that it’s fine to use
callwhen it is appropriate. However, I agree with the article that it’s not something you should be doing “just because you can”. If you look at code that instantiates an object and then uses the object as a function, it is more confusing than simply calling a method on the object. It is overly “cute” and non-obvious. In most cases, simple code that does not obscure what is going on is better than code that is slightly harder to follow.Yes, as an experienced Python developer, you should be aware that an object may be callable via magic methods, but it’s way easier (especially for developers less experienced with the language) to follow code that doesn’t rely on such “tricks”. I’m a big proponent of using language constructs well, but I think their use is similar to “big” words in a natural language; you can eloquently express yourself by using them, but if you over-use them, it makes you seem pompous and harder to understand, even for people that have a large vocabulary. No everyone is at the same level you are. Sure, you can look it up in a textbook, just like you can look up new words in a dictionary. But not requiring the less fluent reader to do so unnecessarily is a mark of skill for an author.
Of course there are many situations where using callable objects is suitable: like you said the SQLAlchemy DSL, or situations where code requires a callable, but you need to do more bookkeeping than is practical with a closure (which would happen when you’re using an API in a slightly different way than intended). Right tool for the job and all that…
In my opinion code shouldn’t be written poorly so that it’s easier for newbies to follow. If you can make it easier for newbies to follow without sacrificing anything then that’s good, but in this exact case in the article, a method ‘get_name’ is used instead and that name is more confusing than just using
__call__!I didn’t mean to imply that you actively dumb down your code. I just meant that using more “complex” features should be carefully considered and only used when necessary or definitely clearer.
Maybe a better name of the method would be
store_matching_filenameor something like that, because that’s what it is doing. You could also argue that this is a DSL that matches files so it’s fine to use__call__, and if it’s part of a bigger library or system with more such classes that might definitely be the way to go. But for a one-off file matcher I’m not sure I would do this. Of course, it’s quite a contrived example so it’s neither here nor there.Independent of whether to make objects callable for this use case:
get_name()was a puzzling choice for this particular method. Accessor methods often take this kind of name and this is not an accessor. If I needed an accessor with Python, I’d probably just access the attribute or create a@property-decorated method. But if I was reading code that called someget_name()I’d expect it not to mutate and I’d also expect it to return a value.A use case that I like for
__call__is making a class that can be used as both a context manager and a decorator. Pretty handy for things like timing or handling specific errors in a consistent way.I’ve overridden __call (in Lua, not Python, but it applies) but it’s not often I do. In one case, it was a wrapper for
syslog(). The module includes three methods,open(),log()’ andclose(). I really did not like the idea of having to callsyslog.log()all the time, so I made thesyslogobject callable.I have the opposite advice:
https://effectivepython.com/2015/02/12/accept-functions-for-simple-interfaces-instead-of-classes/
It comes down to what best expresses the intent of the code. For stateful closures,
__call__is often the clearest approach.Can someone please explain why is the callback version more appropriate? For me - and I’m guessing for most people, callbacks are confusing.
A callback/closure is more “lightweight” than a full-blown named class. You would also need to instantiate the class, which is not necessary with a closure: you just write it on the spot and it has access to the local variables. In Python, however, closures are a bit confusing or unusual, so I would argue that it might not be the clearest solution even here, especially since it’s arguing about simpler code :)
Closures are no more lightweight than objects.
They’re equivalent, as a Schemer I know that in my bones. However, it certainly feels like a lot more ceremony is involved in Python at least to make an object: you must declare the class and methods on it, then instantiate it, and finally call the method. Defining a closure inline and calling it is more off-the-cuff.
Thank you for the explanations!