1. 14

    Some of the other things you get!

    1. 4

      I recently found the statistics module and was grateful to not have to roll my own stddev.

      1. 1

        Yesterday I found the secrets module, did a little dance, then realised it’s not in py2.7. Dancing stopped, because the project I was working on is mid-port to py3, so needs to support both for now

      2. 2

        There’s definitely mocking in Python 2 - is anything different about the Python 3 one?

        1. 3

          I’m not sure about functionality differences, but it’s part of the standard library in Python 3.

      1. 3

        Look forward to playing with the asynchronous generators and list comprehensions.

        Also the nice simple string formatting :)

        1. 9

          How many different ways can you format strings now?

          value = 'test'
          # 1
          "%s" % value
          
          # 2
          "{}".format(value)
          
          #3
          "{name}".format(name=value)
          
          #4
          f"{value}"
          

          Has pep-20 been forgotten? What happened to “There should be one – and preferably only one – obvious way to do it”?

          1. 9

            (1) should be considered deprecated. I would have loved to see it go away in 3.x, but I assume it remains to support legacy code.

            (2) and (3) are the same mechanism. It’s analogous to *args vs. **kwargs.

            (3) is now just the implementation behind (4). I wouldn’t considering them separate.

            1. 6

              A few things:

              • You forgot the Template system.

              • Numbers 2 and 3 on your list are just variations on format() and are basically the same.

              • The new f"{value}“ syntax appears to be just a shorthand for ”…“.format().

              Anyway, I’ve seen this complaint specifically about Python3 at least twice now, and don’t really understand it. Python2 already had a lot of ways to format strings, so why complain now? A new variation on an existing method isn’t hurting anything.

              As far as PEP-20 goes, having one way to do something is only one item on the list, and I’d say this new system helps with several others, so it’s a trade off. There are a many ways to loop, too, and nobody complains about that. In an ideal world or brand new language I’m sure they’d love to get rid of at least % formatting and maybe even .format(), but after all the whining about things breaking in Python3 they’re probably not in a huge hurry to break something that will impact literally almost everybody using the language.

              1. 4

                You forgot the Template system.

                I purposefully left it out, but you are probably right that it is similar enough to be included in my list.

                Python2 already had a lot of ways to format strings, so why complain now?

                Why not? I complained then too! python3 was supposed to be the time to break things and clean things up. It just seems like an odd wart to have so many ways to do string formatting.

                Comparing this to looping constructs seems a bit disingenuous.

                1. 1

                  Fair enough. I had personally never heard this complaint before except just recently in relation to Python3.

              2. 3

                Sure it’s redundant, but it doesn’t seem like an interestingly harmful kind of redundancy to me, because none of the string formatting mechanisms interact with each other in any interesting way. After the expression is evaluated, you can’t distinguish what kind of format string was used by looking at the value. The vast majority of the time the format string will be a literal constant too, so other code won’t be able to distinguish what type of format string was used prior to the expression being run either.

                It’s not awkward like, say, mixing several different object systems (like we had with ‘old style classes’ vs ‘new style classes’ in Python 2.2 through 2.7) where you have to remember which one has been used, maybe you can mix both in one class’s inheritance diagram, and the differences between the two are actually a little subtle to begin with.

                1. 5

                  I’m with you here. I’ve always found the need to use .format() or % for string interpolation to be needless boilerplate, having come from languages like Ruby or even dare I say it Perl where string interpolation happens out of box.

                  If anything, I find the ‘f’ syntax to be rather elegant because, unlike these other languages, you don’t get bitten by interpolation when you’re not expecting it. In that sense, I find the new syntax to be very Pythonic.

            1. 35

              The killer features for Python 3, IMHO, are the clean Unicode/bytes distinction and the normalization of the object model.

              The problem with Python 3 is, of course, that Python 2 is “good enough”.

              1. 7

                I have spent the last couple of days wrangling with generators and co-routines in Python2. I wish the project I’m working on was started in Python3 (considering it’s 2 years old). Python3 is light years ahead in terms of async programming features baked in.

                1. 5

                  You can start calculating by looking at Python 3.0’s release date (in 2008) and then wonder why it hasn’t been adopted within 8 years. But I think of this differently.

                  The first viable candidate for a Python 3 migration is Python 3.4. It’s the release you pick, when you want to maintain a 2 and 3 compatible code base, which is what many libraries need to do. And it is kind of known in the community that 3.0 had a few regressions in terms of stability and speed over 2.7 (never used 3.0 so this is hearsay, nevertheless even that hearsay would keep me from migrating 2.7->3). Now Python 3.4 was released in 2012. which is basically 4 years ago. 4 years is a long time, but it definitely isn’t 10 years.

                  The problem with Python 3 is, of course, that Python 2 is “good enough”.

                  Indeed, and during Python 3’s development and release cycle, Python 2.7 also saw improvements, patches, etc. which means, that it didn’t immediately started to decay, like any other unmaintained software would.

                  Now, if you run Python on a Desktop, it was definitely possible to use Python 3.4 as it was released. But if you run Python on servers, you are bound by the release schedule of your distribution. So until you migrated from Debian wheezy to Debian jessie, you were essentially bound [$] to Python 2. Coincidentally, 2015 was the year when I started to hear many success stories at Python conferences on the hallway tracks: “We did it”, or “I was surprised how easy it was”. For many, now is the time that many passionate Python devs realize: Ah, I finally could use Python 3, let’s see if my code will work on it. And suprise, surprise it works.

                  [$] And since Python is often used in linux distributions, you are not as likely to manually roll out an official release but stick to the distribution-provided one. If you would run Haskell for example, you would probably have a independent update cycles for your linux distribution and Haskell.

                  1. 5

                    clean Unicode/bytes distinction

                    The author argues it isn’t clean, and it’s actually obnoxious, counterintuitive, and frustrating.

                    In an attempt to make their strings more “international” they turned them into difficult to use types with poor error messages. Every time you attempt to deal with characters in your programs you’ll have to understand the difference between byte sequences and Unicode strings.

                    If we have a string and a bytes type then when we [concatenate them] we get an error, and [using .format()] we get a repr formatted byte string.

                    I agree with the author on this, even if the rest is classic Zed Shaw angry rambling. It’s a huge pain in the ass. Go has a clean unicode / bytes distinction, python3 does not. Or maybe it does, and I’m just a foolish peasant for wanting to write scripts in python without screwing around with unicode.

                    1. 12

                      It’s a strange argument. I own a codebase that works in both 2 and 3, and by the far the most annoying errors I have to chase around are Python 2’s unicode/bytes confusion. Python 3’s hard distinction is a much better experience for me, and leads to cleaner code with fewer “just in case” calls to .decode() and .encode().

                      1. 2

                        Or maybe it does, and I’m just a foolish peasant for wanting to write scripts in python without screwing around with unicode.

                        I have been writing scripts in Python 3 for years now and haven’t experience any issues like this. Can you show an example?

                        1. 2

                          The first script I tried to write in python3 was working with hex, which appears to be a fairly pathological case. As such, I remember it vividly. This is approximately what happened:

                          $ python3
                          >>> 'text'.encode('hex')
                          Traceback (most recent call last):
                            File "<stdin>", line 1, in <module>
                          LookupError: 'hex' is not a text encoding; use codecs.encode() to handle arbitrary codecs
                          >>> import codecs
                          >>> codecs.encode('hex', 'text')
                          Traceback (most recent call last):
                            File "<stdin>", line 1, in <module>
                          LookupError: unknown encoding: test
                          >>> codecs.encode('text', 'hex')
                          Traceback (most recent call last):
                            File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/encodings/hex_codec.py", line 15, in hex_encode
                              return (binascii.b2a_hex(input), len(input))
                          TypeError: a bytes-like object is required, not 'str'
                          
                          The above exception was the direct cause of the following exception:
                          
                          Traceback (most recent call last):
                            File "<stdin>", line 1, in <module>
                          TypeError: encoding with 'hex' codec failed (TypeError: a bytes-like object is required, not 'str')
                          >>> codecs.encode(b'test', 'hex')
                          b'74657374'
                          >>> h = codecs.encode(b'test', 'hex')
                          >>> [h[i:i+2] for i in range(0,len(h),2)]
                          [b'74', b'65', b'73', b'74']
                          >>> ' '.join([h[i:i+2] for i in range(0,len(h),2)])
                          Traceback (most recent call last):
                            File "<stdin>", line 1, in <module>
                          TypeError: sequence item 0: expected str instance, bytes found
                          >>> ^D
                          $ python2
                          
                          1. 2
                            >>> import codecs
                            >>> codecs.encode('text'.encode(), 'hex')
                            b'74657874'
                            

                            End-to-end it’s not all that bad. What would it be like in any other language that actually distinguishes text and bytes?

                            $ python3
                            >>> 'text'.encode('hex')
                            

                            It’s usual for encoding names to be character encodings: UTF-8, BIG5, ASCII. Does hex really fit there?

                            >>> codecs.encode('hex', 'text')
                            Traceback (most recent call last):
                              File "<stdin>", line 1, in <module>
                            LookupError: unknown encoding: test
                            

                            Seems like test and text got a little mixed up here in the transcript.

                            >>> h = codecs.encode(b'test', 'hex')
                            >>> ' '.join([h[i:i+2] for i in range(0,len(h),2)])
                            Traceback (most recent call last):
                              File "<stdin>", line 1, in <module>
                            TypeError: sequence item 0: expected str instance, bytes found
                            
                            >>> h = codecs.encode(b'test', 'hex')
                            >>> b' '.join([h[i:i+2] for i in range(0,len(h),2)])
                            b'74 65 73 74'
                            

                            Beware the FUD.

                            1. 3

                              It’s usual for encoding names to be character encodings: UTF-8, BIG5, ASCII. Does hex really fit there?

                              No, it did not. Python 2 also has a rot13 codec (for the this module I suppose), which was probably also axed.

                          2. 1

                            Same here, I’ve spent a few days tracking down unicode related bugs in Python 2, so far Python 3 has been much better when it comes to unicode. (or maybe I just didn’t hit the edge cases yet)

                        2. 3

                          I would divide Python unicode users into three groups:

                          1. For English speaking beginners and those writing simple scripts that don’t need unicode, Python 2 wins by a mile.
                          2. For non-English speaking beginners, Python 3 definitely wins, but probably not by all that much as most non-English speakers are used to painful internationalization issues (I know, this isn’t fair, but it’s true).
                          3. For projects which have to consider unicode from the start, Python 3 is better. However, the lack of compatibility with older Python 2 libraries is so painful that it cancels out any benefit that you can get from better unicode support.

                          IMO, Python 3 has been a huge failure, and while I’m sure it’s painful for the core development team, it should be dropped sooner rather than later.

                          1. 8

                            As a Korean, I have a problem with this:

                            For non-English speaking beginners, Python 3 definitely wins, but probably not by all that much as most non-English speakers are used to painful internationalization issues (I know, this isn’t fair, but it’s true).

                            This is not true, at least not here. I really can’t imagine how it would be true elsewhere though. If someone is “used to” painful internationalization issues, how can that one be a beginner? To state this the other way, if there are such “know internationalization” beginners, there must be more “don’t know internationalization” beginners.

                            1. 2

                              A beginner who’s used computer programmers is used to having to use an ascii-fied version of their name in popular computer programs. Eevee’s response post gives the example:

                              Hi, my name is Łukasz Langa. .agnaL zsaku�� si eman ym ,iH

                              In my experience any beginning-programming Łukasz who has used a computer for a while is used to Łukasz randomly turning into ��ukasz, and so won’t find it particularly surprising or puzzling that their program does that.

                              1. 2

                                I agree, but I still don’t see how that makes Python 3’s definite advantage compared to Python 2 “not all that much”.

                                1. 2

                                  so won’t find it particularly surprising or puzzling that their program does that

                                  Your argument is “it sucks everywhere therefore we shouldn’t even try to do better”? I think that is sad. And international names might also happen when you’re a native speaker: your form application might be used by a foreigner and then it breaks in surprising ways (kinda like Amazon fails to deliver diacritics). I don’t think that having a ticking time bomb bug in your program is a good state to be in and getting a Python 2 program to be all-Unicode is rather hard.

                                  1. 2

                                    My argument is “it sucks everywhere so doing it better is clearly not something people are that bothered about, in a revealed-preferences sense”. In an abstract sense we should all make our programs better in all respects. Concretely, correct handling of non-ascii should probably be a low priority for language designers, since the available evidence suggests it’s a low priority for language users.

                              2. 3
                                1. For English speaking beginners and those writing simple scripts that don’t need unicode, Python 2 wins by a mile.

                                That is an argument, until you run into some bug because that piece of pure-English text contained some unicode ? symbols. Also, I don’t see how beginners would have a very much time in Python 3 because of the byes vs strings distinction.

                                A definitive change for the worse for beginners was the laziness-by-default changes for many functions. Really makes it harder for them to experiment in the Python prompt.

                                1. 2

                                  To be honest, handling emojis is not a beginner task. Running into bugs is not a beginner task. Teaching resource should be carefully prepared so that learners do not run into bugs.

                                  1. 1

                                    With controlled “input data”, I don’t see how Python 3’s unicode handling would be more hostile to beginners.

                              3. 2

                                But what do you think Python3 is killing?

                                It’s clearly killing python2, but programmers, instead of redeveloping their python2 knowledge inside python3 are considering other, more durable languages.

                                I’ve still got C and perl programs more than twenty years old that are running in production (and on the Internet).

                              1. 2

                                I frankly don’t get why people insist on spaces. Please, someone, enlighten me. Please. I wanna understand why a single resizeable character is inferior to a small pile of unresizeable ones.

                                1. 4
                                  1. tabs (and all other invisible characters, see dos2unix) have consistently proven to be difficult-to-understand and error-prone among computer users
                                  2. resizing is easy in an environment you’re familiar with, but in practice people often operate in environments that they are unfamiliar with, so the lowest common denominator (i.e. what I can do without thinking in Notepad.exe) wins.
                                  1. 5

                                    The biggest problem is that they get mixed inadvertently. Go has shown that a program can actually fix it. As a result, my editor reads tabs, I often type spaces, and gofmt makes it standard tabs on save. This is legitimately the best feature of the Go tool chain.

                                    Years ago when I programmed in Python, there would always be that one committer who would insist on tabs, or use 3 spaces instead of 4 in some places, and all hell would break loose in the commit history–annoying, but not the end of the world. The shameful person, btw, didn’t do the cleanup, cause they didn’t give a shit.

                                    In summary, this is the least of our problems and can be automated incredibly effectively if it means enough to you, but if not, who cares?

                                    1. 1

                                      Nowadays, tooling has improved in Python, too. You can also use tools like autopep8 or yapf (https://github.com/google/yapf) to autoformat Python code (yes, that is a thing ;) )

                                      1. 1

                                        That’s pretty great! Thanks for the link!

                                    2. 1

                                      The problem is the assumption that allowing it to be resized is even desirable in the first place. If you are working on a specific codebase, you need to stick with the style that the codebase dictates. Imagine if people thought it was a good idea to allow “flexible brace styles” to let you use your preference. Why must indentation be different?

                                      If you want the code to be displayed differently according to your preference, it should be implemented as a rendering-time hack (for instance, showing “lambda” as “λ”, etc) rather than changing the actual bytes on disk. Rendering 4 spaces on disk as 8 spaces on screen is a trivial config change in any decent editor.

                                      1. 1

                                        Some languages are whitespace sensitive and a tab doesn’t always represent the same number of characters so making everything done with spaces is the simple, easy way out when dealing with vertical alignment. Gofmt does vertical alignment pretty nicely and it works well across different editors and even with doublewidth characters involved etc but most developers don’t like using a formatting tool outside of what its editor does on its own and most languages don’t even have an agreed upon format so there lacks formatters for those languages, because adoption would stumble.

                                        1. 1

                                          Also, if you follow a convention like “maximum line-length 80 chars”, …

                                          1. 1

                                            In order to allow tab characters in your code, you have to disallow certain coding styles to prevent misalignment when tab sizes change. It’s much easier to simply disallow all tab characters, especially as the team size increases.

                                          1. 2

                                            What does lobsters think about Django? I rarely hear it mentioned, but I’ve tried it on a couple small toy projects and it seemed nice enough. The only thing that bugs me is that I’m not sure I understand it’s model. It doesn’t seem to be MVC, and the whole concept of “apps” is foreign to me. Does anyone have a good description of how the Django paradigm is supposed to work?

                                            1. 4

                                              It’s considered heavyweight; it has a unique design pattern paradigm. It’s sort of a Major Thing that your entire stack is built around. Flask is thinner and is preferable for smaller projects.

                                              I’d rather use Rails, myself; Python feels like its getting pushed into only the scientific computing community over time.

                                              1. 4

                                                I primarily use django for backend development when it is a work related project. I transitioned away from django rendering templates to it serving a REST api using django rest framework. I find it pleasant to work with, test and debug but that might come from familiarity. I personally like the ORM and try to avoid writing any SQL in django projects by hand. For some smaller things I have used flask which is a more ‘lightweight’ framework. I think django is in somewhat of a transition period away from trying to do everything in the MVC model to allowing you to make more decisions, looking at the roadmap and announcements the addition of things like channels and more REST features seems to show that it is moving to what I would loosely call a ‘modern web’ approach.

                                                1. 2

                                                  I used django in the past (admittedly a long time ago), and I recall it working well. I remember the documentation being very good. I have also heard good things about the django rest framework from a couple of past colleagues.

                                                  That said, I personally tend to prefer more minimalistic options like falcon, flask, or bottle. I am also not a big fan of ORMs in general, but I have heard of people using django and writing sql by hand too – ymmv I assume.

                                                  1. 2

                                                    I’ve used Django professionally and inherited code from a Django creator. It’s pretty similar to Rails, but built on the deep assumption that your app has a lot of public read-only users (who are probably not logged in) with a small, completely-trusted set of read/write admins. This is really obvious in the built-in admin panel, but it echoes through the form builder, views, routing, templating, caching, apps, etc. I didn’t recognize it while working on a CRUD app with lots of read/write users, but as soon as I touched a publishing site it was like the whole thing sung in sympathetic resonance with a background noise I only just realized I’d been hearing the whole time.

                                                    “Apps” are what they sound like. Django reifies the concept because of the hassle that went into standing up a webserver to serve python apps in the early 2000s and because it was built for the case where your same trusted admins publish multiple sites.

                                                    1. 1

                                                      I thought the same for a long time, however I’ve come to realise this is probably more of a shortcoming in documentation than anything. Django has a great built in permissions system out of the box, the thing it lacks is a simple way to manage those permissions without using the admin page or rolling one yourself. You can set read/write/edit/delete permissions on models or, with a library, instances of the models as well.

                                                      An project I ran for a while at the university was a discussion board similar to stack overflow for students. We had multiple permission levels (students, TAs, course staff etc) and all of that was managed within Django. Coming back to Django rest framework they have a good permissions model as well which I use quite extensively these days to restrict actions / visibility of data based on user attributes.