1. 54
  1. 14

    Really surprised fstrings are faster than string.format. I thought it was just syntactic sugar for that.

    1. 18

      From disassembling both versions, it looks like Python added a new bytecode, FORMAT_VALUE, specifically for f-strings, while format is a plain function call. I’d guess most of the savings is skipping the function call (Python function calls are not fast).

      >>> dis.dis('f"{x} {y}"')
        1           0 LOAD_NAME                0 (x)
                    2 FORMAT_VALUE             0
                    4 LOAD_CONST               0 (' ')
                    6 LOAD_NAME                1 (y)
                    8 FORMAT_VALUE             0
                   10 BUILD_STRING             3
                   12 RETURN_VALUE
      
      >>> dis.dis('"{} {}".format(x, y)')
        1           0 LOAD_CONST               0 ('{} {}')
                    2 LOAD_METHOD              0 (format)
                    4 LOAD_NAME                1 (x)
                    6 LOAD_NAME                2 (y)
                    8 CALL_METHOD              2
                   10 RETURN_VALUE
      

      Edit: Digging a bit more, f-strings were initially implemented as syntactic sugar for a call to format, but that was changed due to semantic concerns rather than speed: If a user redefined the format function, the behavior of f-strings would change, which was considered strange/undesirable. So a new opcode was added to avoid f-string behavior depending on the current binding of format, with a side bonus of being faster. I guess format can’t compile to that new opcode precisely because user code is allowed to redefine it.

      1. 5

        I mucked about with this a lot for a silly optimization in a web framework and concluded that it’s not any specific opcode that makes f-strings faster, it’s fundamental to the design (the design it’s settled on now; no idea about the history). (Although, of course, loading format for each value might be bad enough to make the whole thing consistently slower than other options).

        So it’s not exactly hard to be faster than str.formatstr.__mod__ (or "%s%s" % (foo, bar)) is also often signfiicantly faster. There are two reasons for this: special method lookups are faster than named method lookups, and %-formatting is inherently faster than {}-formatting due to its simpler parser and non-generic architecture. Getting back to f-strings, they can be faster than either because not only is there, as you’ve observed, no method call, there’s almost no (runtime) implementation to call either. Processing the f-string happened at compile time. FORMAT_VALUE takes care of formatting values, with a hardcoded shortcut for values that are already strings, while BUILD_STRING just concatenates a bunch of pieces.

        But although f-strings can be faster than the other two, they aren’t always. They are pretty much always faster than str.format, but %-formatting sometimes beats them; the simplest case that I know where this is true is formatting a bunch of ints:

        $ python3 -m timeit -s 'a, b, c, d = 1, 2, 3, 4' '"%d%d%d%d" % (a, b, c, d)'
        200000 loops, best of 5: 1.59 usec per loop
        $ python3 -m timeit -s 'a, b, c, d = 1, 2, 3, 4' 'f"{a}{b}{c}{d}"'
        200000 loops, best of 5: 1.97 usec per loop
        

        This happens because of that non-generic architecture thing. %-formatting has to parse at runtime, but its parser is very fast. On the other hand, since it just innately knows what ints are, it doesn’t have to ask its operands how to format themselves. This also means it can just print the formatted int out directly into what will become the resulting string, while the f-string version needs to put it in a temporary string of its own to be concatenated by BUILD_STRING later.

        1. 1

          I find it a little surprising that % formatting sometimes wins because it needs an extra memory allocation for the tuple for the arguments.

          1. 3

            It does, but the f-string version needs 4 extra allocations (for “1”, “2”, “3”, “4”).

        2. 2

          Perl probably would have solved a similar problem by having the sugar compile to a call to CORE::GLOBAL::format rather than looking up format in the current scope.

          Then again, Perl would have solved this specific problem by having an op for string formatting. :)

      2. 7

        Ok, f-strings just jumped to the head of my list of candidates for one of those huge vulnerabilities that affects 40% of the internet.

        1. 8

          But why?

          1. 6

            f-strings are not eval workhouse, this is literally the same as passing user data to .format.

            1. 2

              Which makes it a perfect candidate for the next ‘oh, did I need to sanitize my data?’ vulnerability. SQL injections, XSS, and so on typically all start with a primitive of a simple API that lets you embed user data in some other string and then pass it to some other code that makes some assumptions about it. Anything that makes it easier to do this without thinking is a good candidate for vulnerabilities.

              SQL injections largely went away in non-terrible languages / frameworks / systems by removing string concatenation entirely and providing format-string-like APIs and by making people use them because they were faster (they could pre-compile the query if it’s a constant string and just provide the query identifier + arguments to the RDBMS) and easier to use (the SQL could be in a single place). Anything that makes combining data from different sources into a string easier is likely to make lazy or inexperienced programmers reach for that instead of a tool that guarantees proper escaping.

              1. 2

                Well put. Notice the word “powerful” too. Which is, in my opinion, symptomatic.

                Powerful as in including many features with intricate behaviours that most developers are not aware of. Rather than having a well defined straightforward scope.

          2. 4

            I’ve never been a fan of printf or f strings or any string substitution stuff like this. I can’t possibly be the only person who reads this:

            print(f"{f'${number:.3f}':>10s}")
            

            And thinks it is completely, unfathombly unreadable, am I?

            Is that really preferable to something less magical that takes more lines of code?

            1. 9

              I really like f-strings, but I would not write that. I’d rather break that out into a couple lines.

              1. 3

                I prefer format strings because it makes it easier to “see” what the string is going to look like, as opposed to a bunch of concatenation, probably spread across lines. That being said, you’ve got some weird nested stuff going on there that I probably wouldn’t use, so maybe I partially agree?

                1. 2

                  You can use f-strings to write readable things as well.

                  logger.info(f"Folder for {show.name} is missing, creating...")
                  

                  Instead of

                  logger.info(f"Folder for " + show.name + "is missing, 
                  creating...)"
                  

                  The latter example is not valid Python but you get the point.

                  1. 4

                    Nitpick: You should probably not use fstrings for logging, since I believe they’re interpolated every time, even if your log level is below that.

                    1. 3

                      Ah, thanks! I hadn’t thought of that.

                      1. 2

                        Wouldn’t the concatenation also happen every time, though? Logging functions are just regular functions, so their arguments are evaluated before the function is called, right?

                        1. 1

                          Yes. Here’s how to avoid unnecessary interpolation:

                          logger.info("Folder for %s is missing, creating...", show.name)
                          

                          Most linters will warn about this.

                          1. 2

                            I’d forgotten all about that! Haven’t used Python logging in a bit :-) Thanks!

                  2. 4

                    Doesn’t work without JS… I assume because of the stupid loading circle and Google Analytics?

                    Edit: Oh my bad, they decided to write the whole fronted of their blog in JS. What a world we live in.

                    1. 5

                      You also need to allow port 8080. You know, like a normal page of interweb text. 🙄

                      1. 4

                        If I were to pastebin a few KiB of text, would you run it on your machine in an interpreter of my choice? It would be quite reasonable to turn down my offer, I think!

                        1. 3

                          I think it’s okay to have fun with your webpage! He wrote it that way because he wanted to play around with Vue.

                          1. 4

                            I admit my previous comment was rant-y. I think I just woke up on the wrong side of the bed today.

                            There is nothing wrong with at all. You should do what you want with your own website. But on that same token, surely some opposition is expected regardless of anything you post on the internet. In fairness, I didn’t make the best comment. I instead should’ve been constructive and said that perhaps he should consider a non-JS fallback, and given my POV of a” lot of websites use tons of JS, but only a small few actually completely bar you out for not having JS enabled”.

                            I dunno where I’m trying to go with this. So I’ll sum up with, at least don’t bar people from your website.

                            Edit: I mean “you” as in a general sense. Not like you, you, specifically.

                            1. 5

                              You have nothing to admit, you are 100% right. This website requires you to run code on your machine to display text. That’s really a problem.

                              On the other hand, if someone is willing to add so much complexity on top of a simple web page just to display text, then I probably don’t need to heard what that person has to say about software development.

                              1. 14

                                This whole comment thread is 1) off topic 2) boring and overdone 3) unnecessarily harsh toward the author. Read the article or don’t, we don’t need to dredge up this same tired argument every time a link to a JS-heavy blog gets posted. I thought it was an interesting article and I was looking forward to seeing the discussion but the majority of comments are about JavaScript.

                                1. 7

                                  There’s a rather important point here : if you want your message to be heard, then make sure do not distract the attention with a fancy/hard to open envelope. If you want to message to be heard, always choose the most simple, direct way to send your message. (I feel that every single developer should be reminded of that regularly).

                                  The reason why it is important to tell it every time, especially on a website dedicated to developers, is that maybe, just maybe, the original author is young and just doesn’t know there’s something else than JS and Chrome and Google. Maybe the author doesn’t know that there are many really valid reason to disable JS or not being able to run it. Education about diversity is utterly important in a world dedicated to monopolies. So those comments are needed.

                                  Just ask yourself : how did multiple people realize there was a problem with this site ? Not because we inspect the code source of every single website but because we were surprised to not see any content here and investigated to see if there was a problem on our side. Because the title was appealing but, for whatever reason, we were locked out of the content. And how do you want people to talk about a content they can’t read?

                                  I admit I was a bit harsh against the author in my previous comment. I assumed a kind of malevolent behavior from someone perfectly knowing what he/she was doing. I’m apologize for that because maybe the author simply didn’t know and reading those comments might be a small learning experience.