1. 5

    I feel like the author’s points are very muddled because of the need to attack the term “object-oriented” as designating some particular style of code or language design. Python is object-based, which means that even when using data classes and functions, one is still using objects.

    To make this clearer, let’s translate the example code into a flavor of E. E does not have classes but every value is an object.

    def authorAPIMaker(constructURL):
        return def makeAPIClient(rootURL :Str, sessionMaker :SessionMaker):
            return object self:
                to getItems(entity :Str):
                    def resp := requests<-get(`$rootURL/${constructURL(entity)}`)
                    return when (resp) ->
                        [for n in (resp.json()["items"]) makeItem(n)]
                to saveItems(entity :Str):
                    sessionMaker<-withScopedSession(fn session {
    def constructDefaultURL(entity :Str):
        return `/v1/$entity`
    def constructClientAURL(entity :Str):
        return `/$entity`
    def constructClientBURL(entity :Str):
        return `/a/special/place/$entity`
    def clientA := authorAPIMaker(constructClientAURL)("https://client-a", sessionClass)

    Classes (“objects”) are replaced with closures, but all values are objects, including the functions. Inheritance is replaced with composition. The detailed work that the example does is more apparent; we can’t do I/O synchronously, and so we are forced to factor the code so that URL construction doesn’t conflate ownership of the root URL value (which belongs to only the API client) with ownership of the path (which belongs to the URL constructor).

    I think that the biggest difference between the author’s style and the E/Scheme/objects-as-closures style of design is that data classes are antipatterns in E. Rather, we are deliberately bundling state and behavior together, and if only a few behaviors ever need some state, then we should isolate that state within an object and put the behaviors into its methods.

    1. 2

      Thanks for a really clear example.

      Refactoring into closures like this makes it more apparent that my distaste isn’t so much for the object-y sugar-y stuff, but the act of bundling data + functions as a default approach, whether that be in object form or in closure form. There’s a place for it, but due to the simple fact that it’s harder to serialise/print a closure/object - if you can avoid constructing these things, you may as well not bother. (I’d be interested to know how E deals with this).

      I have some big regrets about not spending more time on an example, which I kinda just threw together - I think a lot of the criticism of conflation of concerns in the class are correct. However, I still back myself to refactor most of what gets thrown at me into something as sane/saner :-)

    1. 4

      I have at home Edsger Dijkstra’s source code for the operating system he wrote in 1965

      There was a brilliant programmer at Digitek who had completely novel and now unknown ideas for software development; he never published anything

      You absolute tease Donald. I would email him, but, y’know..

      1. 20

        Seems like it’s time to post the venerable closures-vs-objects koan again. 🕉

        1. 7

          I remember realising that halfway through SICP and 🤯, so good. I put a link to my take on it at the end of the post.

        1. 3

          I think this debate needs a bit more data, so I’m considering making a “call to arms for Python OO enthusiasts” at some point, my plan is:

          • make a github repo, people raise PRs with their code (do I require tests?)
          • their code should be the most idiomatic OO, least refactorable into data + functions
          • no operator overloading specific stuff
          • max 500 lines + I’m only going to look at 5 submissions - I’ve not got all the time in the world
          • I attempt to expunge the OO and present my results - the mob decides

          Is this fair or useful or just dumb?

          1. 3

            I’d play. I mostly agree with your article but think there’s a better case to be made for OO with immutable value objects. Meaning: I think I could write object solutions that would be more or less equivalent to any bag-of-functions approach, both in readability and verbosity.

            1. 1

              I don’t think I could fit a readable submission into 500 lines, but a common (and classical) use for OO is polymorphic evaluation of simple abstract syntax trees for small expression/rule languages. Each different meaningful term or syntax element is a class, and every class has an .Eval() method (usually they take a context object as the argument). To evaluate an expression or a rule, you .Eval() the top level object for the entire rule’s AST, and it .Eval()s its children as appropriate, and so on. Nothing has to know anything about what all of the different terms, operators, elements, and so on are, which makes it quite easy to add more, tinker with the internal implementations, and so on.

              The nicest Python non-OO version I can think of would be to have the AST nodes be bag-of-data objects (different data for different types of nodes) with a common ‘type’ field that named their type. Then you would have a big dict mapping types to their evaluation functions; these functions would take the node bag-of-data object and a context object as arguments. You could also implement a giant ‘eval’ function that knew all of the node types and how to evaluate each one, but that gets ugly.

            1. 6

              sh is a cute-sy interface for shell scripts in Python. Examples from the docs:

              sh.wc(sh.ls("-1"), "-l")
              p = sh.find("-name", "sh.py", _bg=True)
              # ... do other things ...
              1. 3

                Plumbum is usually the Python shell DSL I see recommended. Haven’t used either though, as subprocess.run is generally good enough.

                1. 2

                  I feel like I’ve written 60% of that library in a less useful way several times. I need to remember that for next time. Thanks!

                  Also, it begs for a sibling that wraps paramiko in an ssh interface.

                  1. 1

                    Doesn’t look as convenient as

                    p = $(find -name sh.py &)

                    [ That was shameless plug, that’s NGS ]

                  1. 1

                    See also FastAPI, which seems to be gaining a lot of traction and is pretty pleasant to use.

                    Is there a production ready Flask + Pydantic + APISpec library doing the rounds? I’m a bit wary of by async by default, at least for the time being.

                    1. 1

                      I’ve just been introduced to instant_api which may be similar enough to the Flask + Pydantic + APISpec library you are looking for.

                      1. 1

                        I don’t understand your wariness; FastAPI is async by default being an ASGI framework built on Starlette. If it is the route handlers, Quart also allows for sync route handlers. Quart runs the sync route handlers on a thread executor - as I think FastAPI/Starlette does.

                        I’m not aware of a Flask extension that works with Pydantic - although you can just strip out the async and await keywords from Quart-Schema and it will work with Flask (Quart is a re-implementation of Flask’s API using async/await).

                      1. 4

                        Neat digging, in a similar vein, PySnooper does the same kinda magic debug printing. See also, IceCream. Would love to see some variant in the stdlib some time..

                        1. 2

                          It’s also similar to q.

                          1. 2

                            PySnooper is probably hooking into the debugging trace functions, that let you hook in a callback before executing every line of code (this is also used by stuff like coverage IIRC).

                            IceCream is clearly very close to my thing though. And it has, like, test cases. Not a super fan of the ic calling convention but game respects game.

                          1. 4

                            While the performance of static pages tends to be dominated by render-blocking network requests

                            In (maybe) the majority of cases I’d say static sites aren’t render blocking. If you go to your devtools and set throttling to 56kbps, then click a link on the orange website with >100 comments, you will see a useful section of the page load well before all of the html is downloaded - browsers doing this kind of trickery outweighs almost all js performance magic in my experience. (I’d be interested in a rundown of these kind of tricks and how to optimise to them if anyone has a link to hand).

                            1. 2

                              The core of it is simple, but not easy: deliver enough information to start rendering. Practically, that means a short block of inline styles, followed by content, with no style or script tags that reference separate resources (put those after the main content).

                              1. 3

                                I wouldn’t overthink this. Just serve a simple HTML-document and refer to an external stylesheet using “<link>” or “<?xml-stylesheet>” (rarely known but really cool) that is shared across the site. Once it has been loaded just once, every consecutive access of other sites yields instant styling, because the browser cached it.

                                If you overthink it with inline styles and orderings, the browser won’t be able to leverage caching, and when you put style declarations at the end the browser has no chance to “invoke” the cached style until the HTML is fully loaded, which might make a difference if you have a really slow connection or a very large HTML-document (e.g. a large table).

                                And even the first load, which I mentioned earlier, won’t get harmed too much by missing external CSS, given browsers are optimized enough to immediately send a request (on an open keep-alive connection, which is the default) for an external CSS as soon as they “read” the -tag. The CSS-data will start streaming in at best after just one RTT, which is just a few ms, making it comparably fast to inline-CSS, with the added bonus of the aforementioned possibility for caching.

                            1. 4

                              For unbalanced teams of multiple players, it might be worth checking out TrueSkill. I think it handles unbalanced teams of multiple players better than Elo does. a nice PDF

                              Great article! I always find that working on something that you and your friends immediately use is super fun, I always use it as a chance to try out something new.

                              1. 2

                                For unbalanced teams of multiple players, it might be worth checking out TrueSkill

                                Thanks for this resource! I might prototype with this later.

                                1. 2

                                  An ex-colleague of mine implemented trueskill on the frontend for a table football scoreboard we used at work. I think he in turn adapted it from a python implementation. If you’re lucky there’s something useful in there!

                                  1. 1

                                    Thank you! This library is super clear and helpful.

                              1. 2

                                I’m attempting to chip away at a bit of this problem via an unopinionated “Modest progressive-enhancement” approach that aims smooth the transition between classic backend-rendered apps and SPAs. I think “have the luxury of operating on a deeply improved web stack” is key here, you can get very far these days without pulling in the kitchen sink.

                                1. 4

                                  Sorry for my ignorance, but is there the intention to support version-based dependency resolution with deno, or will all dependencies just be duplicated?

                                  Eg. if I import two libraries, one uses http://somewhere/underscore/1.1.0/index.js, and the other http://somewhere/underscore/1.2.0/index.js, but both libraries only actually require 1.0.0 < underscore < 2.0.0, will always two copies of underscore be downloaded?

                                  If the answer is yes, this might push people culturally to smaller dependency trees, which may be no bad thing. (As an aside: I suspect the “you can only install one copy of a dependency at once” in python - despite being a technical limitation - has contributed to API stability in libraries).

                                  1. 9

                                    I hope you don’t mind if I take this opportunity to plug my parser, Lark: https://github.com/lark-parser/lark/

                                    It’s written in Python (pro or con? you decide), and supports both Earley and LALR(1), and has a long list of features.

                                    Unlike the package written by the author of the article, it’s not YACC-compatible, and I think it’s for the best :)

                                    It actually improves on the classic LALR(1) algorithm, and can parse things that YACC (or PLY) cannot.

                                    1. 3

                                      I used Lark for a recent project and it was an absolute pleasure, thanks for the hard work!

                                      1. 1

                                        Nice, the implementation looks very clean!

                                        Just one note, you’re using ambiguity="explicit", but doesn’t seem like there’s code to handle an ambiguity (although there probably shouldn’t be one). If it’s just there for debugging, maybe add an _ambig rule to your transformer that throws a nice error.

                                        1. 1

                                          Thanks. You’re correct about not correctly handling _ambig - it should throw an error.

                                          As author of Lark, what are your thoughts on the ideas in this blog post?

                                          Vis-a-vis my project: before I build something with it, I’d like to make the error handling better, and improve the performance - do you think these are both feasible whilst still using Lark?

                                          1. 2

                                            Re performance - absolutely. You’re using the Earley algorithm, which is strong but pretty slow. If you switch to LALR you will see a big improvement in performance (something like 10x if not more), and it should be capable of parsing your language. You might not even need to change that much of your grammar.

                                            Re errors - I’ll be honest, I didn’t read the article that deeply. But the examples it gives at the end (in the table), are definitely things you can do with Lark. Check out this example to see how: https://github.com/lark-parser/lark/blob/master/examples/advanced/error_reporting_lalr.py

                                            Error recovery is also possible. It’s still a new feature, but it’s already usable, and I believe it will become even better soon. See this example: https://github.com/lark-parser/lark/blob/master/examples/advanced/error_puppet.py

                                      2. 2

                                        Thanks for Lark!

                                        I’ve used Lark in the past to parse (a subset of) the LLVM-IR.

                                        1. 2

                                          Oh dang, lark is awesome, thanks for building and maintaining such a great project. The ability to swap LALR and Earley without having to change the grammar and the terse way of notating the grammar to elide unused nodes to keep the parse tree clean are pretty killer.

                                          1. 2

                                            I used Lark for a take-home interview question recently and it came in super handy. Thanks for your work on it!

                                            1. 1

                                              Does it support error recovery (general context-free error recovery as in Aho et al.) by any chance?

                                              1. 1

                                                It supports manual error recovery, not automatic. The article you linked to sounds interesting, unfortunately I’ll have to pay to view it and find out if it’s actually interesting.

                                            1. 1

                                              As a Python dev, there are two aspects to OOP that I see that are if not always bad, at least complicated:

                                              • It encourages clumping together functions and data. This gets particularly hairy when data is being mutated.
                                              • It encourages the use of the more “advanced” syntactic bits of python.

                                              Almost all the places I see OOP used, it becomes an intractable mess pretty quickly, where “plain data” plus a “bag of namespaced functions” would have solved the problem just fine.

                                              Having said that, I think there are use cases where the tractability (i.e. simply being able to follow the code) is outweighed by having a cutesy interface - this is in the case of super generic, super well-tested, libraries. As an example, even though following the SQLAlchemy source is a nightmare, it’s worth it for an interface that maps so well to the domain.

                                              1. 2

                                                I really enjoyed this, thank you.

                                                Either type of lock can be defeated, but each requires a different large, bulky tool which is useless against the other.

                                                I wonder if Sheldon was assuming here that a U-lock is typically defeated with a portable jack?

                                                1. 15

                                                  Sheldon Brown died well before portable angle grinders became common.

                                                  1. 5

                                                    I’m glad you enjoyed it and thanks for saying so.

                                                    I agree that he’s probably referring to an airjack. It could be that was common practice among thieves when the article was originally written. All the thefts I have personally heard of in the last few years have used angle grinders - obviously you can tell by looking at the remains of the lock what was used.

                                                    1. 2

                                                      I’ve bust open (my own) D-lock with a bog standard car jack - makes a satisfying bang when it blows. Interestingly, I did this on a busy Saturday on Slough (a notoriously rubbish UK town) high street, and received no attention whatsoever..

                                                    1. 3

                                                      A semantically aware diff tool for Python that would work in tandem with git to aid refactoring work.

                                                      Instead of a load of line changes you would get eg. “moved function get_foo from bar.py to baz.py”

                                                      1. 3

                                                        I hacked on a Date-inspired relational DSL for a bit that addressed some of the things in this article. It included relation literals so you could do like:

                                                        v :left_id :l :foo
                                                        v not_foo
                                                            | :right_id | :left_id | :r |
                                                            | 1         | 1        | 11 |
                                                            | 2         | 1        | 12 |
                                                            | 3         | 2        | 23 |
                                                        ^ :new_header value 9

                                                        Warning: the interpreter was v much written in a hurry.

                                                        1. 3

                                                          Looks like about 8 years ago that the creator of Flask/Jinja2 published Classy.js, which is an open source implementation of this idea. I had vendorized it on a couple of projects because it’s just 160 lines of JavaScript and “cleaned up” classes in ES3.

                                                          Docs: https://github.com/mitsuhiko/classy/blob/master/docs/index.rst

                                                          Code: https://github.com/mitsuhiko/classy/blob/master/classy.js

                                                          It’s a good way to grok JS’s object/prototype nature. Of course if you want JS classes today, you probably just want to use ES6 classes. But as this article explains, it’s nice to realize that “classes aren’t magic” and this is probably also why ES6 classes can be babel’ed back to ES3 without issue.

                                                          1. 1

                                                            Good to know Armin’s way ahead of me, love his approach (and his blog)!

                                                            I think the intended reader of this post was me circa 6+ years ago, writing classes as “classes are good”, but not really understanding what they were. It’s part of a series where I try peel back a bit of the magic on a few bits n bobs.

                                                          1. 15

                                                            We had a syntax for HTML-in-JS in the early 2000s. Standardized as E4X in ECMA-357 but never properly implemented by the most popular browsers and then removed. https://developer.mozilla.org/en-US/docs/Archive/Web/E4X.

                                                            Example from https://en.wikipedia.org/wiki/ECMAScript_for_XML:

                                                            var sales = <sales vendor="John">
                                                                <item type="peas" price="4" quantity="6"/>
                                                                <item type="carrot" price="3" quantity="10"/>
                                                                <item type="chips" price="5" quantity="3"/>
                                                            for each( var price in sales..@price ) {
                                                              alert( price );
                                                            delete sales.item[0];
                                                            sales.item += <item type="oranges" price="4"/>;
                                                            sales.item.(@type == "oranges").@quantity = 4;
                                                            1. 2

                                                              If Apple and (mostly) Microsoft hadn’t killed ES4 back in the 2000s, only to re-invent it and call it TypeScript 10 years layer, we’d all be a lot better off by now.

                                                              1. 1

                                                                Ah, good old E4X.

                                                                We had JavaScript style sheets for a while in the old days, too, but they didn’t catch on.

                                                                1. 1

                                                                  Interesting, surprised I’ve never seen this!

                                                                  As someone that wasn’t a professional dev during peak XML, this provokes a similar response to seeing eg. XPath in the wild - “wow, this looks super powerful but maybe a tad too baroque”

                                                                  1. 3

                                                                    For me, E4X isn’t peak XML. It’s the period after the peak where we realised that SOAP was too complex and we were trying to transition away from it to something that was more web-friendly, but had to rely on XML representations.

                                                                    There were definitely people using the full gamut of XML, including XPath, but for many developers the trend was towards SOAP because automatic serialization/deserialization machinery existed and the developer requirement was getting smallish amounts of data from here to there right now.

                                                                    Developers using .NET could just point at a WSDL file and interact with a service without know anything about the underlying mechanisms. This promised interoperability with Java, but was excruciatingly painful because the tools were close, but not close enough, in terms of compliance. Over time, people forgot the full breadth of what XML could do since you weren’t really working with XML at the level where XPath could be used.

                                                                    XMLSpy and other specialist XML tools appeared near the end of the 20th century, but usage dropped, except for niches like publishing and special processes like FDA submissions. Web services took over and morphed into classic SOA. The originators of these were pushing the WS-* standards (that were never intended for human consumption) while implementers and the internet envisioned a simpler model building on REST and JSON. Some people couldn’t get to JSON without an intermediate step of XML, hence E4X.

                                                                    1. 1

                                                                      For me, E4X isn’t peak XML. It’s the period after the peak where we realised that SOAP was too complex and we were trying to transition away from it to something that was more web-friendly, but had to rely on XML representations.

                                                                      Interesting prospective. For me E4X is part of that period where XHTML was starting to be a thing (a desirable thing) but was killed by draconian error handling.

                                                                      All that aside, lest not forget that E4X is also a legacy of ActionScript, the JavaScript-like language that is (was) used to write Flash programs. The ability to seamlessly interact with HTTP endpoints that returned XML data made Flash (at least in the eyes of Adobe) a viable platform for business environments and not just for games.

                                                                      1. 2

                                                                        XHTML arrived at the peak, but it took a long time before people really understood what it meant and tried to apply it 2002-2003 (approx.) It was desirable in a number of ways, but the W3C mishandled the transition because they were staunch XML fans, and the implementers where relatively unresponsive to the market. The WHATWG appeared on the scene in 2004 and the rest is history.

                                                                        E4X grew from the identification of some of the better perspectives at the time rather than being an outgrowth from the XML juggernaut. In those times Flash might still have been considered an option for the next era of the Web (aka. Web 2.0.) Very quickly you’d see Adobe identify that something like Flex was needed, only for Flash to collapse because as a runtime there was never any real outreach/integration with the web. While I don’t think the implementation of Flash would have succeeded for much longer than it has done, I think Adobe made many strategic failures which resulted in a shorter fight. The death knell was Apple, but people on the other side weren’t even going to advocate for a closed platform like Flash.

                                                                    2. 1

                                                                      XPath itself I don’t think is too baroque. At least, if you have to deal with XML, it’s very handy to have on hand, provided that XML document isn’t huge.

                                                                      XSLT would be my point of “eh, this feels like too much”. But Xpath on its own can be very handy.