Threads for ZicZacBee

  1. 1

    This is mostly about JSON-LD being used for namespaces but when I saw this I thought that microservices returning JSON-LD, combining those frarements, using a reasoner, and then querying on the graph would be interesting. Logistics and unusually involved access control come to mind.

    1. 2

      Xah Lee is eccentric to say the least, but I think he’s right about the need for Emacs modernization: http://xahlee.info/emacs/emacs/emacs_modernization.html

      His ErgoEmacs project is the single biggest boost to Emacs usability I’ve seen. It has an external maintainer now: https://ergoemacs.github.io/

      But it needs more work IMHO. Not being an Emacs expert, I can’t even get the Github version to install. I had to use Xah’s old version.

      But this, I feel, is the way to win more users: to round off Emacs’ weird 1970s UI and bring it in line with basically every other editor in the last 25-30 years.

      1. 2

        I agree. I think that there should be an ergo mode that is part of the default set of packages. It should make things as close as possible to what is expected for UIs these days. ctrl-xcv cut and paste, ctrl-z undo. ctlr-s for save. ctrl-f for a search. Probably some sort of region selection that isn’t mark and point.

        It could probably detect if ctlr-x was followed by any key that would be bound to a command in normal emacs and put a warning in the mini buffer about it.

        Install an additional binary where this mode is on by default. Call it emacspad or something.

        1. 1

          Sounds good, although a whole new binary seems like overkill.

          I think one thing that stood out to me in the ErgoEmacs docs on Github was that it said:

          « To enable emacs, add the following to your ~/.emacs or ~/.emacs.d/init.el file: »

          I am a new user. I don’t have either of these files. I don’t know which to use or how to tell.

          Surely this is only relevant to existing Emacs users, who have such a config, and the sane default would be “this is on unless the user has an existing config file”…?

          When I suggested this approach in a mailing-list discussion, the Emacs fans were apopletic and at least one of them said he was blocking me for my vile unreasonable attitude.

          And yet, they wonder why more new users don’t adopt their editor. ¯_(ツ)_/¯

          Problem 2: the instructions assume US English and don’t clearly describe how to configure anything else. I’d think a sane default was “if there is no existing config file, check what the OS regional settings are and use that.”

      1. 15

        There are often these articles that attempt to create a mystique for Lisp. They should be take in context. 25 years ago Lisp was filled with ideas that didn’t exist in a lot of other languages. Perl and Python borrowed ideas from it which lead to many people being exposed to dynamically typed languages where you can do functional style programming.

        The only point in this article that is unique to Lisp is s-expressions.

        The section “Dynamically Typed” is not well informed. It gives a flavor of the time where type systems didn’t deliver much on their promises and the feelings of distrust that inspired.

        If you use Python and work with generators, functools, itertools, async, decorators, duck typing, PEP 484 types, contextlib, werkzeug debugger and I guess lists, dicts and objects, you have seen a lot of the parts of Lisp that got people excited 25 years ago. None of the long time Lispers are going gaga over these features but you don’t need to feel like you are missing out on the mystique of Lisp.

        A Lisp is a valid choice if you want to broaden your exposure to different ideas but it’s 2021 and there are a lot of interesting things going on.

        1. 18

          Pattern matching has been available in functional programming languages for decades now, it was introduced in the 70s. (Logic programming languages expose even more expressive forms, at higher runtime cost.) It obviously improves readability of code manipulating symbolic expressions/trees, and there is a lot of code like this. I find it surprising that in the 2020s there are still people wondering whether “the feature provides enough value to justify its complexity”.

          (The fact that Python did without for so long was rather a sign of closed-mindedness of its designer subgroup. The same applies, in my opinion, to languages (including Python, Go, etc.) that still don’t have proper support for disjoint union types / variants / sums / sealed case classes.)

          1. 45

            Pretty much every feature that has ever been added to every language ever is useful in some way. You can leave a comment like this on almost any feature that a language may not want to implement for one reason or the other.

            1. 14

              I think it makes more sense in statically typed languages, especially functional ones. That said, languages make different choices. For me, Python has always been about simplicity and readability, and as I’ve tried to show in the article, at least in Python, structural pattern matching is only useful in a relatively few cases. But it’s also a question of taste: I really value the simplicity of the Go language (and C before it), and don’t mind a little bit of verbosity if it makes things clearer and simpler. I did some Scala for a while, and I can see how people like the “power” of it, but the learning curve of its type system was very steep, and there were so many different ways to do things (not to mention the compiler was very slow, partly because of the very complex type system).

              1. 22

                For the record, pattern-matching was developed mostly in dynamically-typed languages before being adopted in statically-typed languages, and it works just as well in a dynamically-typed world. (In the ML-family world, sum types and pattern-matching were introduced by Hope, an experimental dynamically-typed language; in the logic world, they are basic constructs of Prolog, which is also dynamically-typed – although some more-typed dialects exist.)

                as I’ve tried to show in the article, at least in Python, structural pattern matching is only useful in a relatively few cases

                Out of the 4 cases you describe in the tutorial, I believe your description of two of them is overly advantageous to if..elif:

                • In the match event.get() case, the example you show is a variation of the original example (the longer of the three such examples in the tutorial), and the change you made makes it easier to write an equivalent if..elif version, because you integrated a case (from another version) that ignores all other Click() events. Without this case (as in the original tutorial example), rewriting with if..elif is harder, you need to duplicate the failure case.
                • In the eval_expr example, you consider the two versions as readable, but the pattern-version is much easier to maintain. Consider, for example, supporting operations with 4 or 5 parameters, or adding an extra parameter to an existing operator; it’s an easy change with the pattern-matching version, and requires boilerplate-y, non-local transformations with if..elif. These may be uncommon needs for standard mathematical operations, but they are very common when working with other domain-specific languages.
                1. 1

                  the change you made makes it easier to write an equivalent if..elif version

                  Sorry if it appeared that way – that was certainly not my intention. I’m not quite sure what you mean, though. The first/original event example in the tutorial handles all click events with no filtering using the same code path, so it’s even simpler to convert. I added the Button.LEFT filtering from a subsequent example to give it a bit more interest so it wasn’t quite so simple. I might be missing something, though.

                  In the eval_expr example, you consider the two versions as readable, but the pattern-version is much easier to maintain. Consider, for example, supporting operations with 4 or 5 parameters, or adding an extra parameter to an existing operator;

                  I think those examples are very hypothetical – as you indicate, binary and unary operators aren’t suddenly going to support 4 or 5 parameters. A new operation might, but that’s okay. The only line that’s slightly repetitive is the “attribute unpacking”: w, x, y, z = expr.w, expr.x, expr.y, expr.z.

                  These may be uncommon needs for standard mathematical operations, but they are very common when working with other domain-specific languages.

                  You’re right, and that’s part of my point. Python isn’t used for implementing compilers or interpreters all that often. That’s where I’m coming from when I ask, “does the feature provide enough value to justify the complexity?” If 90% of Python developers will only rarely use this complex feature, does it make sense to add it to the language?

                  1. 3

                    that was certainly not my intention.

                    To be clear, I’m not suggesting that the change was intentional or sneaky, I’m just pointing out that the translation would be more subtle.

                    The first/original event example does not ignore “all other Click events” (there is no Click() case), and therefore an accurate if..elif translation would have to do things differently if there is no position field or if it’s not a pair, namely it would have to fall back to the ValueError case.

                    You’re right, and that’s part of my point. Python isn’t used for implementing compilers or interpreters all that often.

                    You don’t need to implement a compiler for C or Java, or anything people recognize as a programming language (or HTML or CSS, etc.), to be dealing with a domain-specific languages. Many problem domains contain pieces of data that are effectively expressions in some DSL, and recognizing this can very helpful to write programs in those domains – if the language supports the right features to make this convenient. For example:

                    • to start with the obvious, many programs start by interpreting some configuration file to influence their behavior; many programs have simple needs well-served by linear formats, but many programs (eg. cron jobs, etc.) require more elaborate configurations that are DSL-like. Even if the configuration is written in some standard format (INI, Yaml, etc.) – so parsing can be delegated to a library – the programmer will still write code to interpret or analyze the configuration data.
                    • more gnerally, “structured data formats” are often DSL-shaped; ingesting structured data is something we do super-often in programs
                    • programs that offer a “query” capability typically provide a small language to express those queries
                    • events in an event loop typically form a small language
                2. 14

                  I think it makes more sense in statically typed languages, especially functional ones.

                  In addition to the earlier ones gasche mentioned (it’s important to remember this history), it’s used to pervasively in Erlang, and later Elixir. Clojure has core.match, Racket has match, as does Guile. It’s now in Ruby as well!

                  1. 3

                    Thanks! I didn’t know that. I have used pattern matching in statically typed language (mostly Scala), and had seen it in the likes of Haskell and OCaml, so I’d incorrectly assumed it was mainly a statically-typed language thing.

                    1. 1

                      It is an important feature of OCaml.

                      1. 3

                        I am aware - was focusing on dynamically typed languages.

                    2. 7

                      For me, it is the combination of algebraic data types + pattern matching + compile time exhaustiveness checking that is the real game changer. With just 1 out of 3, pattern matching in Python is much less compelling.

                      1. 1

                        I agree. I wonder if they plan to add exhaustiveness checking to mypy. The way the PEP is so no hold barred makes it seem like the goal was featurefulness and not an attempt to support exhaustiveness checking.

                        1. 2

                          I wonder if they plan to add exhaustiveness checking to mypy.

                          I don’t think that’s possible in the general case. If I understand the PEP correctly, __match_args__ may be a @property getter method, which could read the contents of a file, or perform a network request, etc.

                    3. 11

                      I find it surprising that in the 2020s there are still people wondering whether “the feature provides enough value to justify its complexity”.

                      I find it surprising that people find this surprising.

                      Adding features like pattern matching isn’t trivial, and adding it too hastily can backfire in the long term; especially for an established language like Python. As such I would prefer a language take their time, rather than slapping things on because somebody on the internet said it was a good idea.

                      1. 3

                        That’s always been the Scheme philosophy:

                        Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary.

                        And indeed, this pays off: in the Scheme world, there’s been a match package floating around for a long time, implemented simply as a macro. No changes to the core language needed.

                        1. 4

                          No changes to the core language needed.

                          I’m sure you recognize that this situation does not translate to other languages like in this case Python. Implementing it as a macro is just not feasible. And even in Scheme the usage of match macros is rather low. This can be because it is not that useful, but also might be because of the hurdle of adding dependencies is not worth the payoff. Once a feature is integrated in a language, its usage “costs” nothing, thus the value proposition when writing code can be quite different.

                          1. 7

                            This is rather unrelated to the overall discussion, but as a user of the match macros in Scheme, I must say that I find the lack of integration into the base forms slightly annoying. You cannot pattern-match on a let or lambda, you have to use match-let and match-lambda, define/match (the latter only in Racket I think), etc. This makes reaching for pattern-matching feel heavier, and it may be a partial cause to their comparatively lower usage. ML-family languages generalize all binding positions to accept patterns, which is very nice to decompose records for example (or other single-case data structures). I wish Scheme dialects would embrace this generalization, but they haven’t for now – at least not Racket or Clojure.

                            1. 2

                              In the case of Clojure while it doesn’t have pattern matching built-in, it does have quite comprehensive destructuring forms (like nested matching in maps, with rather elaborate mechanisms) that works in all binding positions.

                              1. 2

                                Nice! I suppose (from your post above) that pattern-matching is somehow “integrated” in the Clojure implementation, rather than just being part of the base macro layer that all users see.

                                1. 2

                                  I think the case is that Clojure core special forms support it (I suppose the implementation itself is here and called “binding-forms”, which is then used by let, fn and loop which user defined macros often end up expanding to). Thus it is somewhat under the base layer that people use.

                                  But bear in mind this is destructuring, in a more general manner than what Python 2.x already supported, not pattern matching. It also tends to get messy with deep destructuring, but the same can be said of deep pattern matches through multiple layers of constructors.

                      2. 8

                        I agree about pattern matching and Python in general. It’s depressing how many features have died in python-ideas because it takes more than a few seconds for an established programmer to grok them. Function composition comes to mind.

                        But I think Python might be too complicated for pattern matching. The mechanism they’ve settled on is pretty gnarly. I wrote a thing for pattern matching regexps to see how it’d turn out (admittedly against an early version of the PEP; I haven’t checked it against the current state) and I think the results speak for themselves.

                        1. 6

                          But I think Python might be too complicated for pattern matching. The mechanism they’ve settled on is pretty gnarly.

                          I mostly agree. I generally like pattern matching and have been excited about this feature, but am still feeling out exactly when I’ll use it and how it lines up with my intuition.

                          The part that does feel very Pythonic is that destructuring/unpacking is already pretty pervasive in Python. Not only for basic assignments, but also integrated into control flow constructs. For example, it’s idiomatic to do something like:

                          for key, val in some_dictionary.items():
                              # ...
                          

                          Rather than:

                          for item in some_dictionary.items():
                              key, val = item
                              # ...
                          

                          Or something even worse, like explicit item[0] and item[1]. So the lack of a conditional-with-destructuring, the way we already have foreach-with-destructuring, did seem like a real gap to me, making you have to write the moral equivalent of code that looks more like the 2nd case than the 1st. That hole is now filled by pattern matching. But I agree there are pitfalls around how all these features interact.

                        2. 2
                          for i, (k, v) in enumerate(d.items(), 1): pass
                          

                          looks like pattern matching to me

                          1. 2

                            Go aims for simplicity of maintenance and deployment. It doesn’t “still don’t have those features”. The Go authors avoided them on purpose. If you want endless abstractions in Go, embedding Lisp is a possibilty: https://github.com/glycerine/zygomys

                            1. 5

                              Disjoint sums are a basic programming feature (it models data whose shape is “either this or that or that other thing”, which ubiquitous in the wild just like pairs/records/structs). It is not an “endless abstraction”, and it is perfectly compatible with maintenance and deployment. Go is a nice language in some respects, the runtime is excellent, the tooling is impressive, etc etc. But this is no rational excuse for the lack of some basic language features.

                              We are in the 2020s, there is no excuse for lacking support for sum types and/or pattern matching. Those features have been available for 30 years, their implementation is well-understood, they require no specific runtime support, and they are useful in basically all problem domains.

                              I’m not trying to bash a language and attract defensive reactions, but rather to discuss (with concrete examples) the fact that language designer’s mindsets can be influenced by some design cultures more than others, and as a result sometimes the design is held back by a lack of interest for things they are unfamiliar with. Not everyone is fortunate to be working with a deeply knowledgeable and curious language designer, such as Graydon Hoare; we need more such people in our language design teams. The default is for people to keep working on what they know; this sort of closed-ecosystem evolution can lead to beautiful ideas (some bits of Perl 6 for example are very nice!), but it can also hold back.

                              1. 3

                                But this is no rational excuse for the lack of some basic language features.

                                Yes there is. Everyone has a favorite feature, and if all of those are implemented, there would easily be feature bloat, long build times and projects with too many dependencies that depend on too many dependencies, like in C++.

                                In my opinion, the question is not if a language lacks a feature that someone wants or not, but if it’s usable for goals that people wish to achieve, and Go is clearly suitable for many goals.

                            2. 3

                              Ah yes, Python is famously closed-minded and hateful toward useful features. For example, they’d never adopt something like, say, list comprehensions. The language’s leaders are far too closed-minded, and dogmatically unwilling to ever consider superior ideas, to pick up something like that. Same for any sort of ability to work with lazy iterables, or do useful combinatoric work with them. That’s something that definitely will never be adopted into Python due to the closed-mindedness of its leaders. And don’t get me started on basic FP building blocks like map and folds. It’s well known that Guido hates them so much that they’re permanently forbidden from ever being in the language!

                              (the fact that Python is not Lisp was always unforgivable to many people; the fact that it is not Haskell has now apparently overtaken that on the list of irredeemable sins; yet somehow we Python programmers continue to get useful work done and shrug off the sneers and insults of our self-proclaimed betters much as we always have)

                              1. 25

                                It is well-documented that Guido Van Rossum planned to remove lambda from Python 3. (For the record, I agree that map and filter on lists are much less useful in presence of list comprehensions.) It is also well-documented that recursion is severely limited in Python, making many elegant definitions impractical.

                                Sure, Python adopted (in 2000 I believe?) list comprehensions from ABC (due to Guido working with the language in the 1980s), and a couple of library-definable iterators. I don’t think this contradicts my claim. New ideas came to the language since (generators, decorators), but it remains notable that the language seems to have resisted incorporating strong ideas from other languages. (More so than, say, Ruby, C#, Kotlin, etc.)

                                Meta: One aspect of your post that I find unpleasant is the tone. You speak of “sneers and insults”, but it is your post that is highly sarcastic and full of stray exagerations at this or that language community. I’m not interested in escalating in this direction.

                                1. 7

                                  less useful in presence of list comprehension

                                  I’m certainly biased, but I find Python’s list comprehension an abomination towards readability in comparison to higher-order pipelines or recursion. I’ve not personally coded Python in 8-9 years, but when I see examples, I feel like I need to put my head on upsidedown to understand it.

                                  1. 6

                                    It is also well-documented that recursion is severely limited in Python, making many elegant definitions impractical.

                                    For a subjective definition of “elegant”. But this basically is just “Python is not Lisp” (or more specifically, “Python is not Scheme”). And that’s OK. Not every language has to have Scheme’s approach to programming, and Scheme’s history has shown that maybe it’s a good thing for other languages not to be Scheme, since Scheme has been badly held back by its community’s insistence that tail-recursive implementations of algorithms should be the only implementations of those algorithms.

                                    You speak of “sneers and insults”, but it is your post that is highly sarcastic and full of stray exagerations at this or that language community.

                                    Your original comment started from a place of assuming – and there really is no other way to read it! – that the programming patterns you care about are objectively superior to other patterns, that languages which do not adopt those patterns are inherently inferior, and that the only reason why a language would not adopt them is due to “closed-mindedness”. Nowhere in your comment is there room for the (ironically) open-minded possibility that someone else might look at patterns you personally subjectively love, evaluate them rationally, and come to a different conclusion than you did – rather, you assume that people who disagree with your stance must be doing so because of personal faults on their part.

                                    And, well, like I said we’ve got decades of experience of people looking down their noses at Python and/or its core team + community for not becoming a copy of their preferred languages. Your comment really is just another instance of that.

                                    1. 8

                                      I’m not specifically pointing out the lack of tail-call optimization (TCO) in Python (which I think is unfortunate indeed; the main argument is that call stack matters, but it’s technically fully possible to preserve call stacks on the side with TC-optimizing implementations). Ignoring TCO for a minute, the main problem would be the fact that the CPython interpreter severely limits the call space (iirc it’s 1K calls by default; compare that to the 8Mb default on most Unix systems), making recursion mostly unusable in practice, except for logarithmic-space algorithms (balanced trees, etc.).

                                      Scheme has been badly held back by its community’s insistence that tail-recursive implementations of algorithms should be the only implementations of those algorithms.

                                      I’m not sure what you mean – that does not make any sense to me.

                                      [you assume] that the programming patterns you care about are objectively superior to other patterns [..]

                                      Well, I claimed

                                      [pattern matching] obviously improves readability of code manipulating symbolic expressions/trees

                                      and I stand by this rather modest claim, which I believe is an objective statement. In fact it is supported quite well by the blog post that this comment thread is about. (Pattern-matching combines very well with static typing, and it will be interesting to see what Python typers make of it; but its benefits are already evident in a dynamically-typed context.)

                                      1. 4

                                        and I stand by this rather modest claim, which I believe is an objective statement.

                                        Nit: I don’t think you can have an objective statement of value.

                                        1. 4

                                          Again: your original comment admits of no other interpretation than that you do not believe anyone could rationally look at the feature you like and come to a different conclusion about it. Thus you had to resort to trying to find personal fault in anyone who did.

                                          This does not indicate “closed-mindedness” on the part of others. They may prioritize things differently than you do. They may take different views of complexity and tradeoffs (which are the core of any new language-feature proposal) than you do. Or perhaps they simply do not like the feature as much as you do. But you were unwilling to allow for this — if someone didn’t agree with your stance it must be due to personal fault. You allowed for no other explanation.

                                          That is a problem. And from someone who’s used to seeing that sort of attitude it will get you a dismissive “here we go again”. Which is exactly what you got.

                                      2. 4

                                        This is perhaps more of a feeling, but saying that Rust isn’t adopting features as quickly as Ruby seems a bit off. Static type adoption in the Python community has been quicker. async/await has been painful, but is being attempted. Stuff like generalized unpacking (and this!) is also shipping out!

                                        Maybe it can be faster, but honestly Python probably has one of the lowest “funding amount relative to impact” of the modern languages which makes the whole project not be able to just get things done as quickly IMO.

                                        Python is truly in a funny place, where many people loudly complain about it not adopting enough features, and many other loudly complain about it loudly adopting too many! It’s of course “different people have different opinions” but still funny to see all on the same page.

                                        1. 3

                                          It is well-documented that Guido Van Rossum planned to remove lambda from Python 3

                                          Thank you for sharing that document. I think Guido was right: it’s not pythonic to map, nor to use lambdas in most cases.

                                          Every feature is useful, but some ecosystems work better without certain features. I’m not sure where go’s generics fall on this spectrum, but I’m sure most proposed features for python move it away from it’s core competency, rather than augmenting a strong core.

                                          1. 1

                                            We have previously discussed their tone problem. It comes from their political position within the Python ecosystem and they’re relatively blind to it. Just try to stay cool, I suppose?

                                            1. 6

                                              I really do recommend clicking through to that link, and seeing just what an unbelievably awful thing I said that the user above called out as “emblematic” of the “contempt” I display to Python users. Or the horrific ulterior motive I was found to have further down.

                                              Please, though, before clicking through, shield the eyes of children and anyone else who might be affected by seeing such content.

                                          2. 5

                                            To pick one of my favorite examples, I talked to the author of PEP 498 after a presentation that they gave on f-strings, and asked why they did not add destructuring for f-strings, as well as whether they knew about customizeable template literals in ECMAScript, which trace their lineage through quasiliterals in E all the way back to quasiquotation in formal logic. The author knew of all of this history too, but told me that they were unable to convince CPython’s core developers to adopt any of the more advanced language features because they were not seen as useful.

                                            I think that this perspective is the one which might help you understand. Where you see one new feature in PEP 498, I see three missing subfeatures. Where you see itertools as a successful borrowing of many different ideas from many different languages, I see a failure to embrace the arrays and tacit programming of APL and K, and a lack of pattern-matching and custom operators compared to Haskell and SML.

                                          3. 1

                                            I think the issue is more about pattern matching being a late addition to Python, which means there will be lots of code floating around that isn’t using match expressions. Since it’s not realistic to expect this code to be ported, the old style if … elif will continue to live on. All of this adds up to a larger language surface area, which makes tool support, learning and consistency more difficult.

                                            I’m not really a big fan of this “pile of features” style of language design - if you add something I’d prefer if something got taken away as well. Otherwise you’ll end up with something like Perl 5

                                          1. 2

                                            What is needed to achieve something like what node or go does? It has venvs for separate library environments, it has pipenv for project local environments, it has venv for project local environments with a specific version of python.

                                            What specific UX changes are needed to assemble these to something similar to what node or go does?

                                            What is done in these other communities to handle the other half of the problem which is the health of library dependencies? Automated testing of all permutations seems impractical for large packages.

                                            1. 3

                                              pipenv/poetry/pdm/etc do achieve npm/cargo/stack/etc style “project” UX. It just took so long… and all the layers are just hidden inside, it’s an abstraction waiting to leak.

                                            1. 7

                                              Here is a useful description of vgo from Cox with the details if you aren’t familiar with vgo and want to follow along with the discussion: https://research.swtch.com/vgo-principles

                                              1. 5

                                                I see a lot of talk about the clickbait title but looked at the tools this guy is pushing? It is basically a hyperlinked printf of all state changes of all variables of the program execution. And it has a tree of values that contributed to each state change. It’s like it printf’ed everything and has a nice viewer.

                                                1. 13

                                                  A bit of a shallow argument against Docker, there are still a lot of unanswered questions about managing the application. How are the VMs that the go binary is deployed to managed? How is the networking configuration and service discovery managed? How do you know what the current version deployed is? How do you manage potential cgo dependencies?

                                                  Maybe the application is simple enough that these questions don’t matter, however more complicated go services need to answer these questions. I’m not advocating for Docker specifically here, I actually use NixOS and systemd to handle a lot of these issues. I just don’t think it’s fair to say that scping a go binary is equivalent to a running a Docker image.

                                                  1. 18

                                                    I kind of agree, but from my experience a lot Docker users are using it for packaging Python applications + dependencies and nothing more. A service manager/systemd and Rust/Go binaries could probably cut some operations overhead.

                                                    1. 35

                                                      a lot Docker users are using it for packaging Python applications

                                                      I think what happened is that pip+virtualenv/ruby+bundler/node+npm have dropped the ball so spectacularly that nowadays the idea of a single executable file you could deploy is seen as some advanced high-end feature instead of … literally the most basic thing you could ask of a compiler.

                                                      1. 4

                                                        Sadly, IMHO, it’s not seen as but it really is. And there is no overlapping between the heavy users of Python/Julia/R/Scala¹ and the opportunity to have a single executable file to deploy. Due to the nature of those languages, the solution is “bring everything you need with you” and it is easier to do it in a container than fiddling with the other solutions.

                                                        ¹ yes, I am aware that you can provide a jar and use the JVM and of Scala Native (that is still under development and without multi-threading the last time I looked).

                                                        1. 2

                                                          Python, Ruby and Node all started when any comparable compiled language was much harder to use. People wanted ease and speed of development. Go is helping with this story, being simpler than C++ and its ilk, but those three interpreted languages are well established and people figured ways around deficiencies (i.e., Docker).

                                                          1. 1

                                                            Python and Ruby both had compilers years (maybe decades?) before Docker became popular, and Node has been based around a JIT compiler (V8) from the very beginning. They just happened to be compilers which ignored that use case for whatever reason.

                                                        2. 5

                                                          I suspect a lot of those folks are inadvertently getting a dev environment that approximates their production. Lots of developers are running MacOS or Windows locally while deploying to a Linux.

                                                        3. 14

                                                          I just don’t think it’s fair to say that scping a go binary is equivalent to a running a Docker image.

                                                          It kind of is, because Docker doesn’t do service discovery, or set up your network, or do anything else really except … run a binary. Oh, and docker {push,pull}. People use tools built on top of Docker to handle all of these things. Docker is little more than a glorified chroot with cgroups and built-in curl to DockerHub.

                                                          the application is simple enough that these questions don’t matter

                                                          Many multi-million dollar companies run on 2 or 3 servers, or a handful. These are not “simple” in the sense that they’re simple toy things, they’re only “simple” in the sense of uncomplicated. Turns out you don’t really need to have a very complex infrastructure to do useful things.

                                                          1. 5

                                                            I’m not advocating that docker should be used here, I’m just saying that there are real problems that it solves. Like I mentioned, I use NixOS to solve most of these problems. For my current project have a single box with a declarative network, postgres, Caddy, and Elixir release setup and a dns entry. I’m not even saying you need this, having an Ansible or Packer setup would work too.

                                                            If they had mentioned any of those tools I would have been fine. Just having an scp to box setup is not enough when the box dies and you need to setup another one, but can’t figure out how the firewall rules worked.

                                                            1. 3

                                                              You’re talking about completely different things than what this article is about. Why says they’re not using some sort of scripted solution to set up a server?

                                                              1. 3

                                                                OK so based on the comments, they don’t have a database or any state. I guess the point of the article is that golang binaries are easier to run than Python applications assuming only static asset dependencies? Or that simple applications are simple to deploy?

                                                                I would agree with that and also that it doesn’t warrant docker. I could do something similar with Java using jpackage. However as soon as you have a cgo dependency, like go-sqlite, you might not be able to just scp the binary.

                                                            2. 1

                                                              I mean OK it’s a glorified chroot with cgroups, but it’s chroot with cgroups. Like great! a single command instead of N commands! Plus there’s like… tagging and stuff, so you’re able to manage it with stuff that’s familiar (built-in version control, which is better than a lot of early-stage infra).

                                                              Even when you just have 3 servers, having an easy way to carry the config reliably for a fourth, or testing that config locally, or all the other things, is very useful. And yeah, Docker doesn’t capture everything, but its big-ball-of-mud strategy means that it’s hard to miss a lot of the common issues with trying to get a new server running IMO.

                                                            3. 12

                                                              Managing VMs, network config, service discovery, current version, dependencies, In most cases those are all handled outside of docker. Or that’s what I’m seeing. Are your experiences different?

                                                              1. 1

                                                                Getting the current version and managing native dependencies are definitely handled by docker. If you’re on a single box and use docker compose, network config and service discovery are handled declaratively and can be replicated by any dev on the team. You don’t have to keep a doc somewhere with a port map or worry about them conflicting with something running on your local machine. If you’re on multiple boxes, then yeah those things aren’t handled though there are many tools that provide those.

                                                                I guess my point about the VM config is that the surface area of configuring the VM is much lower when everything is in docker. Sure it doesn’t help with actually provisioning the VM or its config, but you can reduce the configuration to setting up docker and a firewall rather than needing to install the correct version of postgres, nginx, and any other deps.

                                                                Edit: I’ll admit that I should have put developer setup as a point for docker over service discovery, which doesn’t make much sense.

                                                              2. 4

                                                                Maybe the application is simple enough that these questions don’t matter

                                                                If people.l would pragmatically think what they really need to do, as in what the task at hand is, that would almost always be the case. The application is simple enough.

                                                                1. 2

                                                                  One specific thing that is much easier in containers vs. VMs is initial configuration. With VMs you use stuff like cloud-init which (in my limited experience) is quite painful and hard to test. With containers it’s just “here’s an env variable” or “here’s a file that gets dropped in the right place.”

                                                                  1. 1

                                                                    How are the VMs that the go binary is deployed to managed?

                                                                    That’s a question one should not forget about when using Docker as well. In the end it doesn’t change much. Either DIY (rare with Docker now for various reasons), or you use a finished solution. There is many. There is also overlaps. For example you can use Nomad, the exec driver (if you want isolation) or the raw exec (if you don’t nee) or you use the Docker one for a Docker image, or the JVM one which is great if you want to do something similar with fat JARs.

                                                                    But Docker isn’t the thing that helps you with deployment in the end, but in most instances some separate tooling.

                                                                  1. 2

                                                                    Just focusing on the Docker part of the story, but this feels like “Docker is a VM” type of mindset, instead of “Docker is an application container”. The whole distro should/must be irrelevant if I want to package some Python tool.

                                                                    1. 1

                                                                      Can you describe some tools or techniques that achieve that?

                                                                      1. 2

                                                                        Away from computer, but I would use Nix, Guix or Pex (never used the last one but my ex manager was claiming that it’s the only true path).

                                                                    1. 23

                                                                      How about, “Four books professional server-side Python developers should read (and a few they don’t need to)”.

                                                                      As a client-side C++ programmer, that list doesn’t do me much good, except for “High Performance Browser Networking” which is a killer read for anyone involved in HTTP-based APIs on either end of the wire. (And OK, I do write some utility scripts in Python.)

                                                                      On the other hand, “Design Patterns” is a stone classic.

                                                                      1. 9

                                                                        The stone cold classic is SICP, to the point of cliche but no less valid for it. It doesn’t have the instant for-my-job practicality boost that a style guide like “90 Specific Ways to Write Better Python” does, but it’s in another ballpark of depth.

                                                                        @eatonphil I am curious if you had criticisms of it or if you think its lessons are just not nuts-and-bolts enough for your list.

                                                                        1. 5

                                                                          It’s a perennial goal of mine to finish SICP. 😀 If someone is coming to this list and sees value in four top books, I can’t really recommend a book in the same category here that’s taken me years to finish. But for people who read a lot, of course you can give it a go.

                                                                          I’m very much a PL nerd. As mentioned elsewhere, I’ve spent a lot of time programing in Standard ML (see ponyo on my github) and Scheme (see bsdscheme on my github). Forcing myself to spend years getting good at both has seriously improved my ability to write simple code.

                                                                          But I also can’t really recommend that to most devs either because who knows if you’ll enjoy it. Doing what inspires you is the most important thing so you actually complete what it is you want to do.

                                                                          1. 4

                                                                            Scheme and the art of programming by George Springer is almost the same material as SICP but in Scheme. Honestly if you have worked for years in ML or Scheme you probably don’t need SICP since it is a introductory level text.

                                                                            1. 2

                                                                              I just picked this one up a few weeks back and have slowly been working through it. I’m surprised that I hadn’t run into it before then. It’s a textbook and it reads like a textbook, there’s no getting around that, but it’s accessible and seems to be a great intro to Scheme in general.

                                                                              SICP is using Scheme too, isn’t it? I recall running across MIT being the recommended implementation, but I’ve had no problems with Guile, and there’s a language in Racket, too. I bounce between the Springer book, SICP, and Little Schemer when I’m not too busy. I’m in no rush, but I’ve found those three to be a great way to get acquainted with emacs/geiser & guile.

                                                                            2. 3

                                                                              I’m in the same boat with SICP and similar books with an academic feel. If it’s not project oriented, it’s usually too hard for me to focus on.

                                                                          2. 3

                                                                            I wish I could recommend books on other languages! But no book I’ve read on JavaScript (thinking JavaScript the Good Parts, How JavaScript Works), Go (there’s almost nothing here that’s not just reference), Java (Effective Java is massive and not all useful), or C++ (what, Meyers’ Effective C++?) is in the same category in terms of information density, being well-written, and still being applicable today. I’d be happy to take recommendations though for books not among these.

                                                                            1. 6

                                                                              Thinking Forth (1984) and Writing Solid Code (1993) are two books that radically changed how I approached programming. Yes, one is about Forth, and one is about C, but I’ve found them applicable to more than just those two languages.

                                                                              Edit: added years first published.

                                                                              1. 3

                                                                                Fully agree on both and in particular that TF is applicable beyond Forth. Great book.

                                                                                I wonder whether you might enjoy Elements of Programming and/or The Humane Interface (and here’s the bit I’ve quoted most often). Very different books, both of them insightful and hard. EoP is a C++ book that’s about C++ in the way TF is about Forth, THI is a UI book that uses phrases like “information-theorietic efficiency” instead of “at [famous company] we do this [this way]”.

                                                                              2. 3

                                                                                Personally, my qualm wouldn’t be about the language, it would be about the domain. I haven’t read the books, but those 4 titles really are web focused. I’m sure lots of those translate to other domains, but it’s clearly not aimed as the general purpose developer who doesn’t yet know which domain they will tackle. No, this is for backend web developers.

                                                                                Thus, your title should really be closer to: Four books professional backend developers should read. That way it’s more focused to your actual target audience.

                                                                                1. 3

                                                                                  My point is that it wasn’t intended to be backends focused. I haven’t read any really great books on JavaScript (but HPBN is clearly browser, not backend, focused). And I genuinely wish frontend developers I worked with knew more that’s covered in DDIA.

                                                                                  But I don’t pretend to speak for every domain of programming for sure.

                                                                                  1. 1

                                                                                    It’s easy to for us to get myopic about stuff outside our domain.

                                                                                    Note that even here you seem to be implicitly assuming that not-backend = in-browser. That’s leaving out (native) desktop, mobile and embedded development.

                                                                                    I suspect there aren’t actually any books that are super important for all serious programmers. The books I learned the most from are way out of date (K&R, the Smalltalk-80 Blue Book, Advanced Cryptography) or very domain specific (Unix Network Programming) or both (Inside Macintosh.)

                                                                                    1. 1

                                                                                      Yes I’m definitely missing domains but mobile isn’t one of them! HPBN devotes a ton of time to the effect of protocols on mobile battery life of all things. There’s a chapter or two on the behavior of mobile networks. Mobile developers are also responsible for monitoring and alerting on their production deploys. And they’ll also be heavy users of backend systems.

                                                                                      All this isn’t to say that I’ve got the list perfect. I hope I change my mind on it over time as I’ve read new books, learned new things, and expired different domains.

                                                                                2. 3

                                                                                  I feel The Go Programming Language is quite good; certainly more than just a reference.

                                                                                  The Little Schemer is probably the best programming book I’ve read. While it’s quite limited in scope (functional programming in Scheme), it does explain it in such a way that’s very insightful and useful for other languages too, IMO. And the writing style is just fun!

                                                                                  1. 4

                                                                                    I feel The Go Programming Language is quite good; certainly more than just a reference.

                                                                                    It’s not bad. And certainly new Go developers would do well to use it as a style guide since writing “idiomatic” Go takes a bit. I’ll consider adding it to a future version of this post.

                                                                                    The Little Schemer is probably the best programming book I’ve read. While it’s quite limited in scope (functional programming in Scheme), it does explain it in such a way that’s very insightful and useful for other languages too, IMO. And the writing style is just fun!

                                                                                    In the category of niche languages there is no shortage of gems. :) I haven’t read The Little Schemer but I’m working through The Little Typer at the moment. I’ve got a post in mind for a rundown of 6-8 Lisp books alone. But I wouldn’t consider recommending any of these to developers en masse (even though I believe programming in Scheme or Standard ML for a while helps you write simpler code).

                                                                                    1. 3

                                                                                      I don’t think the value of The Little Schemer is in learning Scheme, but rather in learning recursion and functional programming, and Scheme is just the tool that explains these concepts. The reason I got it in the first place is because it came recommended to better understand JavaScript’s functional aspects.

                                                                                  2. 2

                                                                                    I highly recommend Eloquent JavaScript by Marijn Haverbeke. It has a great style of writing and starts from some really nice computing basics.

                                                                                    1. 2

                                                                                      Learn you a haskell has been transformative to me, for this reason

                                                                                      1. 1

                                                                                        You Don’t Know JS was an excellent language-specific book back in ~2013 when I read it, and I think it’s been updated after that.

                                                                                        I do prefer more general books these days, now that I know enough languages well enough. SICP is a famous one, but I think How to Design Programs might be a more approachable similar book. Nb. I’ve only worked about halfway through it.

                                                                                        1. 1

                                                                                          My copy of Effective Java by Joshua Bloch is 346 pages, are you perhaps confusing it with another? I found it useful, too. Some of it is a bit self-evident, but I opened my copy at a random page and read an item (46, pages 212-214, about for(:) compared to for(;;)) and still found it good.

                                                                                          It’s not a book on java, though. It’s for people who already know java, and who write or maintain large java programs, say 50kloc or more.

                                                                                        2. 3

                                                                                          I dunno, I’m reading through Designing Data-Intensive Applications at the moment, and it feels pretty language agnostic. I’ve seen some snippets of bash, but that’s it. Definitely not Python-focused.

                                                                                          1. 1

                                                                                            Language-agnostic, but server-oriented. That’s why I wrote “server-side”. As a client (mostly mobile) developer, I’m not much concerned with terabyte-scale data or with distributed algorithms that only make sense for clusters in a data center (because they assume latency is zero and uptime is 99% and all peers are trusted.)

                                                                                            I’m not dissing any of this, just pointing out the applicability is narrower than advertised :)

                                                                                        1. 33

                                                                                          Wayland also does not solve any problems for most users. That’s the biggest reason adoption is so slow. People might put up with some breakage for some benfit, but for no (perceivable) benefit? Nah. I run X in all my devices including my phone. I recently had to walk my Mom through switching off wayland (her laptop came with it on) so her screen sharing could work for teaching classes. She hasn’t noticed the switch except that things are reportedly more stable now.

                                                                                          1. 7

                                                                                            That’s completely fair. It is hard to see the benefits of switching when ideally, at the user level, the systems should be indistinguishable.

                                                                                            That being said, I’ve switched because under X screen tearing was driving me nuts, and it has vanished on Wayland.

                                                                                            1. 3

                                                                                              X has absolutely no support for per-screen DPI scaling (and almost certainly never will). It makes my current monitor setup almost completely unusable, so I’m either stuck with Windows or I use wayland..

                                                                                              1. 9

                                                                                                Have you tried xrandr’s –scale ?

                                                                                                xrandr -d :0 --fb 6572x3696 --output DP1-2 --mode 3840x2160 --scale 1x1 --rate 60 --pos 0x0 --primary
                                                                                                xrandr -d :0 --fb 6572x3696 --output eDP1  --mode 1366x768  --scale 2x2 --panning 2732x1536+3840+0
                                                                                                
                                                                                                1. 1

                                                                                                  I was under the impression that DPI scaling is usually done by the UI toolkit (such as GTK) in response to an environment variable that can be set per-process?

                                                                                                  That said, I’ve only playing with DPI scaling on my pinephone. My desktop is a single 4K monitor so no need for any scaling stuff there. I’m very open to believing your use case is one wayland is good at and X is not, in which case by all means :) To each their own.

                                                                                                  1. 1

                                                                                                    That may work for single displays. The issue is two displays (multi-head) with significantly different DPIs. GTK’s scaling works to a degree, but it’s still not per-monitor aware last time I checked, and it doesn’t elegantly handle font scaling.

                                                                                                    1. 1

                                                                                                      GTK’s under X is per-process. So unless windows from one process are on each monitor… then I guess you’re back to the same problem again if you need to scale on one of them.

                                                                                                      Like I said, I don’t use scaling usually. I just run things at their normal size.

                                                                                                      1. 2

                                                                                                        Or if you drag windows between monitors. Also, if you’re using a 4k display on X, you’re probably using a 200% scaling factor (unless you’re on a huge monitor), either as a pixel scale or your desktop is scaling for you.

                                                                                                        1. 1

                                                                                                          Yup, huge monitor :) I wouldn’t bother with 4K on a smaller monitor.

                                                                                              1. 4

                                                                                                This is cool and all but it’s a bit of a downer that strictly worse mainstream languages are adopting features from better languages and implementing them in a strictly worse way. E.g. in OCaml you would get an exhaustiveness error like:

                                                                                                Warning 8: this pattern-matching is not exhaustive.
                                                                                                Here is an example of a case that is not matched:
                                                                                                Scheduled
                                                                                                

                                                                                                Instead of the inscrutable:

                                                                                                error: Argument 1 to "assert_never" has incompatible type "Literal[OrderStatus.Scheduled]";
                                                                                                expected "NoReturn"
                                                                                                

                                                                                                (Also, you wouldn’t need to remember to put an else: assert_never(...) at the end of the code.)

                                                                                                1. 6

                                                                                                  In OCaml pattern matching is a core feature, here it’s being emulated by clever users. There’s a proposal to add pattern matching to Python 3.10, in which case presumably Mypy would be able to do exhaustiveness checking.

                                                                                                  (Also, calling Python “strictly worse” is a bit of a strong statement. Strictly worse how?)

                                                                                                  1. 2

                                                                                                    Yes, eventually–if and when the proposal is accepted, then ships in Python, then gets implemented and ships in Mypy.

                                                                                                    Strictly worse how?

                                                                                                    • Performance
                                                                                                    • Type safety
                                                                                                    • Language design (including modules), e.g. the recently-added controversial walrus operator in Python is something that automatically and naturally occurs in OCaml due to its original design

                                                                                                    But since this is a programming-language-oriented thread, it’s reasonable to assume that we’re comparing mostly language design.

                                                                                                    1. 7

                                                                                                      On the other hand, Python runs natively on Windows, so it’s strictly superior to OCaml as a language.

                                                                                                      (F# is another story…)

                                                                                                      1. 5

                                                                                                        Difficult to tell whether this is a serious assertion, but on the off chance:

                                                                                                        • OCaml has no fewer than three ports to Windows. If this seems like a disadvantage, remember that many people actually recommend Anaconda as an alternative to the official Python installer on Windows.
                                                                                                        • The most well-known port, OCaml for Windows, provides recent compiler versions and almost all opam (equivalent of PyPI) packages.
                                                                                                        • Opam is planning full Windows support out of the box in an upcoming release.
                                                                                                        1. 6

                                                                                                          It was tongue-in-cheek up until I read your response, which made it suddenly an excellent argument about why “strictly superior” is nonsense. All three of your ports need “a Unix-like build environment”. Package manager support is only “planned” in the next version, much as pattern matching is “planned” for 3.10. Comparing it to Anaconda, which is recommended to scientists and data scientists who aren’t programmers, is flat wrong. The only people who could think that OCaml Windows support is as good as Python Windows support, which doesn’t require you to install cygwin, are OCaml partisans.

                                                                                                          Admitting that OCaml has worse Windows support than Python is the easiest thing in the world. If you’re not willing to even do that, why should I trust you about any of your other claims?

                                                                                                          1. 2

                                                                                                            Comparing it to Anaconda, which is recommended to scientists and data scientists who aren’t programmers, is flat wrong.

                                                                                                            Let’s actually look past the marketing blurb and check how Anaconda describes itself:

                                                                                                            ’Anaconda Individual Edition is a free, easy-to-install package manager, environment manager, and Python distribution with a collection of 1,500+ open source packages with free community support.

                                                                                                            ’Anaconda Commercial Edition is the world’s most popular open-source package distribution and management experience, optimized for commercial use and compliance with our Terms of Service.

                                                                                                            ’Anaconda Team Edition is our latest generation repository for all things Anaconda. With support for all major operating systems, the repository serves as your central conda, PyPI, and CRAN packaging resource for desktop users, development clusters, CI/CD systems, and production containers.

                                                                                                            ‘Anaconda Enterprise is an enterprise-ready, secure, and scalable data science platform…’

                                                                                                            (From https://docs.anaconda.com/ )

                                                                                                            So interestingly, they position only their enterprise edition as a data science platform, and the others as general-purpose Python distributions.

                                                                                                            Now we can make the argument that it’s still used mainly by data scientists. But that just proves my point! Data scientists, like most other computer users, are mostly using Windows. So why would they prefer Anaconda, which is (if we ignore the Enterprise edition) just a Python distro? Because it has an easy Windows installer which loads you up with commonly-used libraries! I.e. specific distribution for a specific need, just like the OCaml Windows ports.

                                                                                                            Admitting that OCaml has worse Windows support than Python is the easiest thing in the world. If you’re not willing to even do that, why should I trust you about any of your other claims?

                                                                                                            That’s a non sequitur. I was responding to your claim that:

                                                                                                            Python runs natively on Windows

                                                                                                            …with the implication that OCaml does not. You’re a smart guy, you probably already know that it does and has for a long time. I was just expanding on that and saying that the surrounding tooling is also catching up.

                                                                                                            Also, since when is Windows (a non-free operating system) support a measure of the quality of a programming language? Is this a portability argument in disguise? Are you suggesting in turn that OCaml, which has been ported and released for Apple Silicon, is better than languages which have not, and which due to resource constraints, won’t be for a while? Seems like a nonsensical argument.

                                                                                                            1. 2

                                                                                                              And let’s take at the OCaml for Windows page:

                                                                                                              opam-repository-mingw provides an opam repository for Windows - and an experimental build of opam for Windows. It is work in progress, but it already works well enough to install packages with complex dependencies (like core_kernel) and packages with external dependencies (e.g lablgtk).

                                                                                                              The repository is forked from the standard version. It contains Windows specific patches and build instructions. Especially build related tools (ocamlfind, ocamlbuild, omake, oasis, opam and OCaml itself) were modified, so that most unix-centric build instructions will also work with the native Windows/OCaml toolchain and I can sync the repository with the main repo from time to time (and without too much hassle).

                                                                                                              I don’t know about you, but this doesn’t stir confidence in me that I’ll be up and confidently running OCaml in Windows. Which brings up the question again, what does it mean to be “strictly superior”?

                                                                                                              Also, since when is Windows (a non-free operating system) support a measure of the quality of a programming language?

                                                                                                              What does Windows being non-free have to do with being strictly superior?

                                                                                                              Is this a portability argument in disguise

                                                                                                              Since nobody in this thread has defined what “strictly superior” means, why can’t it be?

                                                                                                              1. 1

                                                                                                                What does Windows being non-free have to do with being strictly superior?

                                                                                                                Non-free operating systems don’t have anything to do with language ‘superiority’ at all.

                                                                                                                nobody in this thread has defined what “strictly superior” means

                                                                                                                ‘Strictly superior’ is a subjective, and therefore, opinionated argument. So to each their own, but to me:

                                                                                                                • Statically typed (meaning, no typechecking at runtime)
                                                                                                                • Pattern matching
                                                                                                                • Exhaustivity checking built in
                                                                                                                • Every object/method/value is not a key/value in a bunch of dicts which can be runtime-patched
                                                                                                                • No nulls
                                                                                                                • Immutable values
                                                                                                                • Expression-oriented syntax
                                                                                                    2. 2

                                                                                                      Fwiw, Java supports that kind of exhaustiveness checking now too, added in JDK 13 as part of switch expressions. Seems to be becoming a more common feature even outside of functional programming.

                                                                                                      1. 1

                                                                                                        That’s the thing–all features from functional programming (starting from garbage collection in the ’50s) eventually find their way into ‘mainstream’ programming languages, all the while FP is called ‘academic’ and ‘impractical’ every step of the way 😂

                                                                                                        1. 1

                                                                                                          You can say the same about OO languages. After all, Simula and Smalltalk were certainly not widely popular. C++ first appeared in 1985, which was a good 25 years after Algol-60 was released. And FP languages have gotten plenty wrong. Haskell’s laziness has been criticized for years, and I’d be happier if I didn’t have to deal with Scala implicits in my code again.

                                                                                                    3. 3

                                                                                                      “Not exhaustive” error messages when dealing with non-trivial types are amazing. I sit there and look at the example trying to figure it out then when I do half the time it is a bug that would take a lot of time to find.

                                                                                                      1. 2

                                                                                                        It’s a much better value proposition to add something like this that provides, let’s be conservative, half the utility, to an existing codebase than it is to switch languages (and ecosystems, tools, and so forth) in order to get the full utility.

                                                                                                        At some point the definitions of “better” and “worse” need to be weighted based on how much work is actually accomplished with a given tool / language.

                                                                                                        1. 2

                                                                                                          Hopefully also weighted by a fair accounting of the amount of bugs prevented by the language.

                                                                                                          1. 2

                                                                                                            Don’t listen to him! It’s a trap! It’s totally not fair to allow OCaml and friends to put the bugs prevented by the language on their side of the ledger because those are several infinite classes of bugs.

                                                                                                        2. 2

                                                                                                          My counterpoint is that OCaml only provides this feature through having a purpose-built ADT for it.

                                                                                                          Mypy (and typescript) generalize these concepts by slowing arbitrary union of types. You can have “string or bytes” instead of “Either String Bytes” (with the wrapping required at call sites)

                                                                                                          From a usability perspective OCamls stuff is way more restrictive and verbose

                                                                                                          1. 1

                                                                                                            I guess it’s possible with a suitable type system, e.g. in Scala you can have an implicit that coercing the value into Either, or in C++ it could work via an implicit constructor. But yeah, to me was somewhat surprising to find myself (as a big fan of types!) enjoying mypy and other gradual typing systems more than ‘stricter’/‘static’ ones.

                                                                                                            1. 1

                                                                                                              Arbitrary unions of types turn out to be a bad idea as soon as you need to deal with generics. Difficult to work with A | B if A and B are generics.

                                                                                                          1. 2

                                                                                                            Hobby code for hobbies is fine. Where I work someone pushed through their own version of this to production. I asked them why they didn’t use a static site generator and they said it wasn’t possible with no additional explanation. Then they went to deploy it and it was a disaster but content administration team had already gotten pulled into rewriting everything as markdown so it was going to be a embarrassment if it failed. People stepped into help. This ended up with each node having needing a half dozen manual deploys of flask apps for every update since the original person didn’t consider the production aspects of their site generator. Then more work was done to fix the painful deploy. Later that person with the site generator left the project.

                                                                                                            Their vision was flask apps in docker containers deployed to kubernetes with helm. During the first 2020 presidential debate I redid the whole thing in mkdocs. Two hours later it was static files that could go anywhere. Want a template from the rest of the site? no problem, want code highlighting, no problem, want put yaml in their .md and have a custom template render that part, no problem, search? plugin + single config line, no problem, site map? single config line, no problem. Many of the existing SSG are concise to-the-purpose solutions.

                                                                                                            1. 3

                                                                                                              I mean, just like with most jobs there are people who are bad at it. You don’t necessarily avoid <field-of-endeavor> projects* just because someone did a crap job of it, you fire them if they continually produce crap, or get them some training, or have them work on something more suited to their skillset.

                                                                                                              I also wouldn’t consider a flask app running in k8s to be a “static site generator” by any means! Yikes!

                                                                                                              *: Assuming they are qualified in the first place - nuclear, medical, cryptography, etc.

                                                                                                              1. 1

                                                                                                                Interestingly, horror stories of less-bespoke software constructs such as frameworks are usually painted as a fault of the user, rather than the approach.

                                                                                                                Software reuse at the largest scope possible is the prevailing orthodoxy still.

                                                                                                                1. 1

                                                                                                                  Yeah, there are varying degrees of InventedHere, NotInventedHere, and a lot of YourHoldingItWrong out there!

                                                                                                                  I guess much like the old adage of never getting fired for buying IBM, you aren’t likely to get fired (or heckled by colleagues) for picking popular framework du jour.

                                                                                                                  But we shouldn’t forget that along with the benefits (time savings, shared effort, etc), there are usually downsides to yielding control of your feature-set to the whims of a third party – features you may not need or desire, and if your needs happen to diverge far enough from the framework orthodoxy, then good luck to you!

                                                                                                                  I’m a bit more “for libraries” I guess, and a bit “framework skeptical” (caveat: I don’t work in the frontend/GUI/UI space, which may influence my leanings)

                                                                                                            1. 2

                                                                                                              I sympathize with this guy. I’ve worked on a project that used a BSD 3 clause license and got picked up by a company and sold as a service. I had left the project by then but I didn’t love it.

                                                                                                              It was at a university and they would not allow us to release it under GPL because they were concerned that if it turned out to be something that could be turned into a business, they would not be able to build a business around GPL licensed software. Once it got picked up by a company, my first reaction is that the university lawyers made a mistake, on the other hand their goals and the projects goals were not aligned and the lawyers achieved their goals.

                                                                                                              1. 4

                                                                                                                The main point of this article is to suggest that exceptions should be used for “ask forgiveness not permission” flow control in the current stack frame to clarify code by replacing if/else statements. The discussion is limited to using exceptions for local flow control. The author does not address in any serious way flow control that passes beyond the current stack frame.

                                                                                                                1. 2

                                                                                                                  I think the point is not to clarify if/else statements but that exception capture the intention more. If I want to cast something to an int, I should try to and deal with failure. If I instead try to check first the implementation of the check might not match the implementation of actual action. This is typical for file handling. If you check if a file exists and then read it you have a race condition. You should try to read it and handle failure.

                                                                                                                  1. 1

                                                                                                                    Yes. Their exception version of the float to int check code handles many more failure cases than the if/else one. This doesn’t mean advice about exceptions as larger scale flow control is incorrect.

                                                                                                                    Regarding file handling, yes it is important to know about O_CREATE or the + file flags in Python and handle errors correctly.

                                                                                                                  2. 1

                                                                                                                    The author does not address in any serious way flow control that passes beyond the current stack frame.

                                                                                                                    Such as?

                                                                                                                    I’m not exactly sure what you’re alluding to here, i.e. are you referring to using catching throws from a thread/process (in a hypothetical language where this would be very easy) instead of queue/channels for communication?

                                                                                                                    1. 4

                                                                                                                      One common criticism of exceptions is that they’re a “hidden” control flow, separate from the execution path you can see in the source file, which imposes an additional and persistent cognitive load on readers. Basically, if you’re using exceptions, any function can return at any expression without your knowledge. I think this is what ZicZacBee was alluding to. Your examples sidestep this criticism by always using unqualified ‘except’ clauses — it’s not possible for exceptions to escape your blocks. But this carries it’s own set of liabilities: what if your float conversion failed due to an out-of-memory issue? Shouldn’t that have a different effect than it being NaN or whatever?

                                                                                                                      1. 1

                                                                                                                        But this carries it’s own set of liabilities: what if your float conversion failed due to an out-of-memory issue? Shouldn’t that have a different effect than it being NaN or whatever?

                                                                                                                        OOM issues are particularly easy to avoid since exceptions for things like OOM and sigkill are rather specific in most languages. E.g. in python one can do except Exception: and catch basically everything besides those.

                                                                                                                        But more generally, yes, this is an issue and I even mentioned it in the article as one of the tradeoffs.

                                                                                                                      2. 3

                                                                                                                        Yes, I’m thinking of throwing as a channel for passing the control to a catch higher in the stack frame for something beyond what could be called ‘error handling’. If writing an Akka like system in Python with exceptions well bets are off. The article covers EAFP in local cases with brief examples. It’s a great way to do it. To use that as a basis to disregard other advice about using exceptions for flow control is poor reasoning. Many practices that are a good fit in a 20 line script do not scale to a larger project.

                                                                                                                        1. 1

                                                                                                                          If writing an Akka like system in Python with exceptions well bets are off

                                                                                                                          I mean, I agree that if you’re writing an actor based system using exceptions as control flow isn’t exactly ideal, but it’s not something that would even cross my mind. The concept of exception shouldn’t even exist in an actor-based paradigm. It’s comparing apples with oranges. I’m talking about imperative programming here.

                                                                                                                          Many practices that are a good fit in a 20 line script do not scale to a larger project.

                                                                                                                          Agree, but I don’t think the particular example you gave is relevant as to why this is the case with exceptions as control flow.

                                                                                                                          1. 1

                                                                                                                            If a larger system uses exceptions for flow control the location of the code that handles the exception is likely to end up based on the call stack. If this buys us something of more value than the cost of the complexity, it could be worth doing.

                                                                                                                            Yes, apples with oranges, we are comparing small imperative code snippets that use exceptions as flow control with a large systems that use exceptions as flow control. One of these it is a good fit and the other it is not.

                                                                                                                    1. 10

                                                                                                                      That reminds me of the time when I realized that I made the grave mistake of making all my tables have the encoding utf8_unicode_ci and thinking they were UTF-8, not knowing that utf8_unicode_ci is a legacy broken encoding that only supports up to 3 bytes per character. How naïve of me to assume that utf8 means UTF-8. I realized it when I tried to insert some 4 byte UTF-8 characters and the database spewed back errors at me. You should always use utf8mb4_unicode_ci [1]. That was my big WTF moment with MySQL. Not surprised that there are more skeletons in the closet.

                                                                                                                      [1] https://stackoverflow.com/questions/766809/whats-the-difference-between-utf8-general-ci-and-utf8-unicode-ci

                                                                                                                      1. 7

                                                                                                                        MySQL is full of WTFs. The most recent one I ran into is that GROUP_CONCAT has a maximum length, when exceeded will just silently truncate the resulting string.

                                                                                                                        Another one that sticks out in recent memory is that it thinks "foo " = "foo" is true (so trailing spaces don’t matter). Oh, and the always fun gotcha that DDL statements aren’t transactional. PostgreSQL was such a breath of fresh air after having dealt so often with the pain of having migrations half-applied and then crashing (while writing them). And this kind of nonsense goes on and on and on.

                                                                                                                        1. 2

                                                                                                                          You’re not the only one. I got caught by this one when someone stuck an emoji in a search query that ended up in a URL.

                                                                                                                          1. 2

                                                                                                                            Having worked with Oracle databases before, this really explains why Oracle the company was happy to buy MySQL - like sees like.

                                                                                                                            1. 1
                                                                                                                            1. 1

                                                                                                                              Play with the kids hopefully a bike ride. Deadlifts. Put some finish on some window trim.

                                                                                                                              1. 19

                                                                                                                                “The root cause is a lack of thinking.” We would all like to appear pro-thinking.

                                                                                                                                I’d suggest the root cause are time constraints.

                                                                                                                                1. 3

                                                                                                                                  A lot of the reason I’m checking Stack Overflow is (1) the docs for tool/framework/library X are horrible (2) this is the umpteeth tool/framework/library/build system/command line tool to do thing Y I’ve used and can no longer keep everything I need in my head.

                                                                                                                                  1. 2

                                                                                                                                    I would like very much to join the “We would all like to appear pro-thinking” party! Or can we get that added to some group’s platform?

                                                                                                                                    Limiting our sophistry to at least appear in favor of thinking is something we should all be able to get behind.

                                                                                                                                    1. 10

                                                                                                                                      We’ll have buttons that say “I’ve thought about it!”

                                                                                                                                      1. 4

                                                                                                                                        I don’t know… I’d at least like to wait until my preferred political party or favourite celebrity endorses this “pro-thinking”. I don’t want to go chasing fads.

                                                                                                                                        1. 3

                                                                                                                                          That could be another party slogan! “I don’t know…. I don’t want to go chasing fads”.

                                                                                                                                      2. 2

                                                                                                                                        and time constraints come from need for profit. The market economy is incompatible with good code.

                                                                                                                                        1. 18

                                                                                                                                          That’s ridiculous. Time constraints can come from anywhere and trying to use it as a dig against market economies is tenuous, at best.

                                                                                                                                          1. 2

                                                                                                                                            Then it might be by chance that the software world is in a terrible state and the more money-constrained enterprises produce the worst software.

                                                                                                                                            Anyway could you tell me where time constraints would come from in the current production mode?

                                                                                                                                            1. 7

                                                                                                                                              A limited lifespan in which to do what we must do.

                                                                                                                                              Many of us are keenly aware of our impending end of file.

                                                                                                                                              1. 4

                                                                                                                                                It’s not obvious to me that more money constrained enterprises do in fact produce worse software in general; or even that software can be ranked on a single good-to-bad axis irrespective of the goals of the developers and users of that software. I don’t see the amount of money an organization has as being particularly related to how good their software is, or even see that as a particularly relevant question, given that an instition might want to use their money to produce software that does what they think is good rather than what I think is good.

                                                                                                                                            2. 6

                                                                                                                                              I generally have a lot more time to spend on code I’ve written while employed than code I’ve written in my spare time.

                                                                                                                                              1. 1

                                                                                                                                                that probably means your company has no pressing need to deliver new and unfinished software and it’s healthy enough not to pressure lower layers in the hierarchy into working crazily even if unnecessary.

                                                                                                                                                1. 4

                                                                                                                                                  My personal experience is that time constraints are rarely the reason for bad software; rather, a number of organisational and/or historical reasons are. Writing bad or good software usually takes about the same time Actually, writing good software is usually faster if you take the long-term view, and I think most people understand that.

                                                                                                                                                  The “write once, never look back” coding CEO, that kinda clueless coworker who doesn’t quite seem to know what they’re doing, that asshole coworker who absolutely insists on rewriting everything in their preferred way as that’s the One True Way, that other team in the other city which was recently hired and doesn’t understand the context, changing requirements or directions from management, the customer, or legislation, unclear requirements, a generally toxic work environment which leaves everyone demoralized, someone deciding we should follow latest fad X. Stuff like that tends to be a far larger influence in my experience.

                                                                                                                                                  Of course, I’ve only worked in five companies, so it may be different in other companies. But this is my experience based on those five companies.

                                                                                                                                                  1. 2

                                                                                                                                                    Or you could accept the market forces at that place converge on “do it better.”

                                                                                                                                                2. 2

                                                                                                                                                  Which explains why all the good software comes from north korea, the soviet union and pre-reform china…

                                                                                                                                                  1. 1

                                                                                                                                                    no but a lot of good software comes from research centers, foundations, public institutions, where the priority is on quality, reliability, correctness, fairness and developers are free to work without being directly exposed to a market-driven feedback loop. That doesn’t mean that these institutions are not immersed in a market economy, but that they invest resources in shielding some people from these pressures in order to be able to produce a kind of quality that is not possible otherwise.

                                                                                                                                                    1. 2

                                                                                                                                                      And these research centers, foundations, public institutions are of course located in societies with free(-ish) markets.

                                                                                                                                                      1. 1

                                                                                                                                                        yeah but they can produce that software despite the market, not thanks to the market. The way they get funding and allocate resources is in response and opposition to the market needs.

                                                                                                                                                        Peer 2 Peer production is also done in societies with free markets, but they work really hard to try to escape it. It’s a bug, not a feature.

                                                                                                                                                3. 1

                                                                                                                                                  Even when I don’t have a time constraint set by some sort of external party, I still get stuck in the trap of searching Stack Overflow and becoming more and more frustrated (although this usually happens with infrastructure and deployment problems, not actual programming issues).

                                                                                                                                                1. 50

                                                                                                                                                  Regardless of whether you currently think your existing tools need replacing, I urge you to try ripgrep if you haven’t already. Its speed is just amazing.

                                                                                                                                                  1. 7

                                                                                                                                                    I’ll second this sentiment. Your favored editor or IDE probably has a plugin to use ripgrep and you should consider trying that too.

                                                                                                                                                    1. 6

                                                                                                                                                      As an experiment I wrote a tiny Go webservice that uses ripgrep to provide a regex aware global code search for the company I work at. The experiment worked so well over a code base of ~30GB that it will probably replace hound which we use for this purpose at the moment. I did not even use any form of caching for this web service, so there is still performance to squeeze out.

                                                                                                                                                      1. 5

                                                                                                                                                        https://github.com/phiresky/ripgrep-all comes with caching, it’s a wrapper around rg to search in PDFs, E-Books, Office documents, zip, tar.gz, etc

                                                                                                                                                      2. 3

                                                                                                                                                        ripgrep and fd have changed the way I use computers. I’m no longer so careful about putting every file in its right place and having deep, but mostly empty, directory structures. Instead, I just use these tools to find the content I need, and because they’re so fast, I usually have the result in front of me in less than a second.

                                                                                                                                                        1. 5

                                                                                                                                                          You should look into broot as well (aside, it’s also a Rust application). I do the same as you and tend to rotate between using ripgrep/fd and broot. Since they provide different experiences for the same goal sometimes one comes more naturally than the other.

                                                                                                                                                          1. 2

                                                                                                                                                            broot is sweet, thanks for mentioning it. Works like a charm and seems super handy.

                                                                                                                                                        2. 1

                                                                                                                                                          3 or 4 years ago it was announced that the vs code “find in files“ feature would be powered by ripgrep. Anyone know if that’s still the case?

                                                                                                                                                          1. 1
                                                                                                                                                        1. 2

                                                                                                                                                          I really enjoyed this, thank you.

                                                                                                                                                          Either type of lock can be defeated, but each requires a different large, bulky tool which is useless against the other.

                                                                                                                                                          I wonder if Sheldon was assuming here that a U-lock is typically defeated with a portable jack?

                                                                                                                                                          1. 15

                                                                                                                                                            Sheldon Brown died well before portable angle grinders became common.

                                                                                                                                                            1. 5

                                                                                                                                                              I’m glad you enjoyed it and thanks for saying so.

                                                                                                                                                              I agree that he’s probably referring to an airjack. It could be that was common practice among thieves when the article was originally written. All the thefts I have personally heard of in the last few years have used angle grinders - obviously you can tell by looking at the remains of the lock what was used.

                                                                                                                                                              1. 2

                                                                                                                                                                I’ve bust open (my own) D-lock with a bog standard car jack - makes a satisfying bang when it blows. Interestingly, I did this on a busy Saturday on Slough (a notoriously rubbish UK town) high street, and received no attention whatsoever..