1. 4

    Why still using twisted in 2020 when asyncio is part of the standard library or you can use more advanced async libraries like Trio or Curio?

    1. 3

      What’s wrong with twisted? It’s battle tested and works well.

      1. 1

        I have been used heavily years ago and I hate him :) The deferred/callback/errback mechanism is very powerful, but the code readability and code maintenance is terrible.

        I find asyncio easier to mantain and I really love Trio

        1. 1

          For a while now you can use async/await with twisted. Makes the code a lot more readable.

          1. 1

            And the article shows that… :)

      2. 1

        Odds are, if it’s a networking or network-related problem, and someone else has encountered it in the past, Twisted will already have the solution either in Twisted itself or in its ecosystem.

        That’s pretty hard to beat.

      1. 11

        Very simple syntax, but I’m not sure I like these constructs. Unless I remember the specification, my knee-jerk reaction is to be surprised and suspicious upon seeing that kind of code. Is it going to be executed or not? Is it a merge conflict resolution gone bad? Especially if there are neighbouring if/else blocks with almost correct indentation.

        Although, I must admit the for/else variant is great for simplifying searches in iterables :)

        1. 6

          This is one of those cases where I think the Python solution is elegant but impractical. I find it much harder to reason about (and consequently I strongly suspect it is more error prone) than (for example) the Go solution to the same problem, which is to continue LABEL (example). That said, I recently screwed up some Go code by typing goto instead of continue, thereby creating a surprising infinite loop.

          1. 3

            I predict that your knee-jerk surprise and suspicion will give way to mild feelings of approval as you get more comfortable with Python idioms. Other languages I’ve known have much stranger idioms, which also become ‘intuitive’ with familiarity. Language is a wonderful thing.

            1. 2

              Is it an idiom (as in, commonly used pattern)? I’ve known about this for years, but never used or seen it in any code.

            2. 2

              In my opinion, the most basic requirement of someone working on Python code is to know Python’s syntax. And in fact that goes for any programming language. One of the best things about Python is that it has really simple syntax for a language of its type (nothing approaching Lisp’s simplicity). It requires only a single token of lookahead to parse, which I think helps humans and computers. There are no “garden path” syntactic constructs where you only realise half way along a line that you need to go back to the beginning and start parsing that line again, like you can get in C++:

              x * y = z;
              

              So when you say ‘Unless I remember the specification’, it makes me wonder whether you actually know the language properly or whether you’re just cargo-culting code based on some sort of surface-level knowledge of the language that you’ve gained only by reading a lot of code. Python’s grammar is right there in the documentation, all these constructs are well-documented and I remember them being covered in the first Python book I ever read, so there’s really no excuse not to be familiar with them.

              1. 1

                I know a lot of people aren’t really fond of Django’s template language, but its for tag uses an optional empty block instead of an else block to clarify that it’s only for things you want to do when the thing you tried to iterate was empty.

                1. 4

                  …which is not the same as what for/else does.

                  1. 1

                    While it’s true that Python’s for/else is not quite identical to the Django template language’s for/empty, handling an empty iterable is the primary use case I’ve seen in the real world for people using Python’s for/else, and avoids some of the confusion of what Python’s for/else actually does (which is, I’d argue, a poor fit for else).

                  2. 1

                    A lot of template engines support this. Actually, I think pretty much all of the ones I’ve worked with over the years across a variety of languages.

                1. 2

                  I once wrote a much smaller version as a tutorial for writing Prometheus exporters :)

                  https://opensource.com/article/19/4/weather-python-prometheus

                  1. 1

                    Pretty cool! I also instrumented the service itself: https://weather.gsc.io/metrics

                  1. 8

                    Gawk has all of these. Don’t port anything.

                    At some point, however, awk’s limitations start to show. It has no real concept of breaking files into modules, it lacks quality error reporting, and it’s missing other things that are now considered fundamentals of how a language works.

                    1. 3

                      GAWK is not portable. You could possibly say “neither is Python”, but I would bet that Python is more available than GAWK. and even if it isnt, if youre going to have to install a package anyway, wouldnt a butcher knife be better than a plastic kids knife?

                      I like AWK, I have used it for many years and would consider myself an AWK expert. But when the question is “what is the right tool for the job”? The answer is rarely AWK or GAWK.

                      1. 10

                        The right tool is obviously Perl in this case.

                        1. 7

                          and the tool is a2p since forever.

                          1. 2

                            Came here to say that very thing. The syntax maps more precisely and the idioms fit more completely thanks to Perl’s history for this almost exact purpose. The right tool for the right job.

                          2. 6

                            What do you mean “gawk is not portable”? Name one platform that has awk and python that does not have gawk?

                            The point is you can either spend your time rewriting or you can just keep using the same code with extensions.

                            And if you really really want to rewrite, Perl is a lot closer. This whole article just seems like someone who has arbitrarily decided that python is a “real” language so it’s inherently better to use it.

                            1. 8

                              The author blurp has:

                              He has been programming Python since 1999

                              Looks like a case of hammer, nail to me.

                              (and the examples with the yields only convince me more that python is not the better choice)

                              1. 2

                                To be fair, I know far more carpenters with Python hammers than with Awk hammers.

                                I myself have but a ball peen Awk hammer, compared to my sledge Python hammer. So for really stubborn nails, Python is the better choice for me.

                                1. 1

                                  I’ve been using Awk for even longer though.

                                  The story in https://opensource.com/article/19/2/drinking-coffee-awk was in 1996.

                                2. 0

                                  Um, Debian, BSD? should I go on?

                                  1. 4

                                    I suppose you mean that gawk features are not portable among the default awk on different OSes, so you shouldn’t use them and pretend that the script will work on any awk. That is totally true.

                                    But the OP likely means that you can use gawk explicitly, treating it as a separate language. Gawk is available on almost all Unix OSes, so it is portable.

                                    1. 2

                                      My point is if your going to have to install a package, you might as well install $proper_programming_language instead of AWK. Unless what you need can be easily done with GAWK alone, its not really worth using.

                                      Keep in mind that even with GAWK, proper support for indexed arrays is not available, nor first class functions, private variables in AWK are footguns, no HTTP client, no JSON, etc.

                            1. 3

                              This doesn’t seem pythonic? idk why you’d appeal to the zen of python if you’re already skipping it.

                              1. 3

                                Explicit interfaces are a form of the “Explicit is better than implicit,” “There should be only one obvious way to do it,” “Practicality beats purity” and “Now is better than never” principles.

                                1. 0

                                  … please google “look before you leap” it’s explicitly rejected by our community.

                                  1. 2

                                    “Explicitly”

                                    “Rejected”

                                    “Our”

                                    “Community”

                                    Cite?

                              1. 1

                                What are the advantages versus something like typing/mypy and Protocol? Mypy has the benefits to not need to use the verify.verifyObject thing and explicitely define who implements what (@implementer).

                                1. 2

                                  You don’t need to use verifyObject. You have the ability to.

                                  You can use interfaces with mypy – there’s a mypy plugin which lets you use interfaces as types: https://github.com/Shoobx/mypy-zope

                                1. 1

                                  If you’re importing from shapes to register implementations of get_area, and them the maintainer of shapes decides to implement a “get_area friendly” new shape class, won’t that cause a infinite import loop? Because area_calculation imports shapes and then shapes imports area_calculation?

                                  1. 0

                                    No.

                                    1. 1

                                      That’s not a very helpful reply.

                                      1. 0

                                        You’re right.

                                  1. 2

                                    One nice thing about doing things this way is that if someone else writes a new shape that is intended to play well with our code, they can implement the get_area themselves

                                    I get why you might use singledispatch to add methods to classes someone else “owns”, but if you were implementing a new class why wouldn’t you just add the methods normally to the class, rather than going through the singledispatch route:

                                    class Ellipse:
                                        horizontal_axis: float
                                        vertical_axis: float
                                    
                                      def get_area():
                                        return math.pi * shape.horizontal_axis * shape.vertical_axis
                                    

                                    This seems a lot more conventional than implementing get_area for the new Ellipse class in the manner the article suggests?

                                    1. 2

                                      This way someone can call get_area(shape) without worrying whether the shape is a circle or an ellipse.

                                      1. 1

                                        Oh 🤦‍♂️ - I totally misunderstood what singledispatch was doing here. Thanks :)

                                    1. 3

                                      So, why not simply extend the class directly? For e.g, given a Circle class,

                                      class Circle(Circle):
                                          def area(self):
                                               ...
                                      

                                      Older Circle instances are not affected (and hence will not do random things if you are redefining a method) and newer instances will answer to shape.area().

                                      1. 0

                                        So…you think that it’s “simpler” to have some Circle objects have “AttributeError” when you call area, and some not. Also, where do you plan to put this code? In the library that’s not yours? How are you going to make sure that all parts of your code generate only newer circles?

                                        I’m not sure the word “simply” means what you think it means.

                                        1. 3

                                          I was not being dismissive of your post, however, you haven’t explained why one should use the specified library. What are the pitfalls of the approach you dismissed as “While it is possible to reach into a class and add a method, this is a bad idea: nobody expects their class to grow new methods, and things might break in weird ways.”.

                                          The approach I outlined is simple, with the caveat I outlined, and I do not think that is a wrong meaning of simple.

                                      1. 3

                                        Missing the essential prequel, “why should a software developer do anything about climate change.”

                                        1. 18

                                          “I don’t see what’s so important about preserving the environment that sustains my life that I should have to do anything about it.”

                                          1. 3

                                            It’s an excellent point and it has nothing to do with Climate Change.

                                            Let’s do a mental experiment. Switch the topic. Let’s say you’re concerned about world hunger.

                                            What can a developer do about world hunger? Well, if you wrote an app that allowed people to sell hot dogs at 20% than they used to, that’d help. How? Because the incremental drop in hot dog prices would affect the entire food chain to some degree, thereby making food cheaper for folks who can’t afford it.

                                            But that doesn’t feel right, does it? Even if overall, your hot dog app actually did more than anything else you could do for world hunger, it doesn’t feel like you’re directly doing something about it.

                                            So, what do you want? What part of this question is about doing stuff, what part is about how you feel about doing stuff, and what part is about making a difference? These are three different issues. Whatever your answer on these issues, you should have some sort of measurable test to see how well you’re doing. Write the test, then make the test pass.

                                            1. 7

                                              That’s a nice case where market forces actually align with ethics. It happens more often than people think about it, since you don’t notice things when they work right. But it’s ultimately a coincidence.

                                              But what about those cases where it doesn’t align? A lot of environmental impact doesn’t show up on the balance sheets, because the cost has been externalized. The obvious example is dumping toxic waste in the river, which might not even affect the employees if they happen to be upstream, but even things like creating a toxic social norm and inducing substance addiction have significant costs that don’t immediately affect the business.

                                              Also, food distribution is pretty well optimized already, except for harmful cultural norms that induce lots of food waste. Those can’t be fixed with app development.

                                              1. 1

                                                Yes, I deliberately tried to keep this very simple as the point was that the complexities expand rather quickly. Folks are welcome to add ethics and other concepts as desired.

                                                This is a very good question because it’s something a lot of people want to know about: I feel strongly about X. What can I do? Far too often the answer given is some sort of self-serving version of “join my cause!” as if everything has already been decided and a few tweets and some donations every month is all that’s needed. Do this, then go on about your life. No further thought required.

                                                It’s an especially powerful question because it needs some decomposition. If you don’t know why you feel strongly about X, in concrete terms, you’re rather unlikely to feel as if you’ve made any progress with it, no matter what you do. How it should be decomposed it up to each person, of course. But it needs work.

                                          1. 3

                                            PyBay! If you’re there, come and say hi!

                                            1. 5

                                              I’m trying to understand if the analysis is normative (“why we should not try to make software secure”) or descriptive (“this is why software security is hard to improve”). If descriptive, then it misses the point explained by many above about externalized cost: even when insecurity ends up costing money, much of the cost is externalized and hard to measure: what is the true cost of the equifax breach?

                                              If prescriptive, then I am pretty sure I disagree. For one, it does incentivize companies to externalize costs of security aggressively. But even more importantly, it ignores a black swan event: a hostile-power-used attack can kill millions, or tens of millions, even if previous breaches only killed hundreds.

                                              1. 2

                                                It would be great to see a follow-up that discusses interface design. In Twisted which interfaces have worked out well, and why? (E.g., I think that IAgent is great while ISevice and IServiceCollection haven’t been so successful).

                                                1. 1

                                                  Wow do I feel personally called out by this! :)

                                                  (Maybe you don’t want a follow-up written by the person who authored IService and IServiceCollection)

                                                  1. 2

                                                    I know you did. :) I thought you might have some perspective. I apologize for not making that context clear.

                                                    To clarify, I mean that I view IAgent as successful because there are many implementations, while folks tend to implement IService and IServiceCollection by subclassing Service or MultiService respectively. So why is that? Is it good/bad/whatever? What does that say about the design of the interface? Composition vs. inheritance etc.

                                                    IProtocol is another interesting case. It’s pretty tiny, but most protocols subclass twisted.internet.protocol.Protocol instead of implementing it directly, I guess for the default method implementations and ILoggingContext. IProtocol is also interesting because due to its complicated relationship with ITransport and the various extension interfaces it is difficult to wrap. Again, a contrast but IAgent, but could you do better? (Is tubes a better design?)

                                                1. 2

                                                  This is describing a value being used in both covariant and contravariant positions.

                                                  To provide a value x : T is covariant. To consume a value (x : T) -> R is contravariant. Covariant uses are unchanged then T becomes S, a looser, less informative type. Contravariant uses are unchanged when T becomes U, a more specific type.

                                                  The reason this doesn’t happen with with functions definitions—which are interfaces themselves—is that every covariant use of that particular interface is controlled by you. Others always use the function contravariantly (they access your single instance and “consume” it via application). This means that you can always make the function more specific by either returning a more detailed output or asking for less of the inputs. The pain of covariant users of this interface is born entirely by you.

                                                  Really there’s no getting around this, but it’s worth thinking about the more general situation and learning the terminology for it.

                                                  1. 2

                                                    I know the terms “covariant” and “contravariant”. Do…do you feel like the blog post would have been clearer or more useful if I dropped those in? Because I am pretty sure that for my target audience, explaining the general situation would have been a distraction.

                                                    1. 1

                                                      Sorry, I think it’s totally fair to just state what you did. It calls to attention a common aspect of the problem.

                                                      I think why I want to invoke the more general terminology and concept is that there’s an implied recommendation—think super carefully about your protocols—with not as much of an offering of the mechanism to do it. The mechanism of “only use protocols you’re willing to support exactly as is forever” is very difficult to use.

                                                      The larger discussion of subsumption is super difficult and probably not something to add in to a post like this, but it’d be great to see it elsewhere.

                                                  1. 4

                                                    oh, man, is anyone else hankering to redo the analysis with Python and Jupyter?

                                                    1. 4

                                                      As always, the proper solution would probably involve violence to astrophysicists.

                                                      1. 5

                                                        It’s a bit more nuanced than that. Astrophysicists don’t use leap seconds.

                                                        The following section on the subject in Wikipedia is rather partisan in tone, but summarizes the recent debate to eliminate leap seconds:

                                                        https://en.m.wikipedia.org/wiki/Leap_second#Proposal_to_abolish_leap_seconds

                                                        1. 2

                                                          What event(s) happened that make that your default reaction?

                                                          1. 4

                                                            Have you ever met an astrophysicist??

                                                            1. 1

                                                              Only briefly, they seemed as nice as the other physicists I’ve met. Or do I not have a large enough sample size?

                                                          1. -1

                                                            This is an interesting article, and possibly quite useful to the people living in the alternative universe where not everything happens via a GitHub PR.

                                                            1. 6

                                                              I find that asking yourself “what am I going to use this for” is critical piece of using names in software. If you have more than one use, you probably want a field per use, with some sort of “default” so if someone does have a name they use for everything, they don’t need to work hard.

                                                              Example of uses:

                                                              • The name that shows you that you’re logged in (e.g., in the top-right corner of a Mac screen or equivalent in web applications) – just let the user pick whatever.
                                                              • The name that social-media apps show to other users – you might need to support features like “report” for this name, so if someone chooses a name like “kill all $ETHNIC_GROUP”, people can use the same report facility as they do for content.
                                                              • The name that your custom support reps will use to greet them. This might need to allow a “Mr./Mrs./Ms./Miss/Mx.” field (probably open text for custom honorifics like “Lord”).
                                                              • The name that you send the payment provider. Check the relevant API for the payment provider, and allow anything the API allows, and nothing that the API doesn’t allow. If the user has a problem, let them take it up with Visa or AMEX.
                                                              • The name that you put on postal notices. It’s reasonable to have a max-limit on the characters here if you need to fit it into a physically constrained space like an address sticker.
                                                              • The name under which they are legally authorized to work in $COUNTRY. They need to put exactly the characters on the work-authorization document that they plan to use to show eligibility for employment. It makes sense to show them the list of possible documents, and tell them to put the name exactly as it appears on that document.

                                                              Here’s the thing: until you do have a use case for a name, don’t ask for it. Once you do, your use case might mean “I’m sorry there’s no way to write your real name, but I need to put something on the snail mail, so put whatever thing would get this to you.”

                                                              1. 5

                                                                I have spent so much time trying to explain the principles you outlined, and how they relate to gender markers and to trans people, to medical professionals over the past few years. (I do this primarily in my capacity as a person who makes noise and tries to push for improvement, rather than as a patient. Sometimes these conversations are made possible because of my employment, and sometimes because of personal connections.)

                                                                I wish it were as simple as just stating those principles and having them be applied. There’s just so much resistance to that. Nonetheless, you’re absolutely correct, and stating the principles clearly is a vital step.

                                                              1. 1

                                                                and the payments file after run the script… how is it updated?

                                                                1. 1

                                                                  After the script runs, you empty the payments file, move the output on top of the old members file, and (if I was doing it now, and not 20 years ago), git commit the result.

                                                                  1. 1

                                                                    I think it’s a ledger that is being updated manually.