1. 3

    I like syncthing, I have used it before, but I’m on an iPhone now and there’s no client for it, so, too bad.

    But what I really wanna say is that I’m glad this is not yet another masturbatory nostalgic rant about how things where great when computers where just shitty terminals running BASIC, or shitty CPUs running DOS, and how the web ruined everything and javascript and npm are the literal antichrist.

    1. 1

      I also looked for an iOS client for Syncthing, and didn’t find one. I had a look at the protocol specification, and it looked like it should be quite doable to implement it as a Files-compatible thing. If only I had unlimited time to work on side projects…

    1. 5

      No, defects are the fault of individual humans and small groups of humans. Defects caused by combining two things that work fine in isolation but not together is the fault of the programmer that integrated them. 95% of the time it’s because they made assumptions along the lines of ‘assume postcondition A implies precondition X’ or just ignored the guarantees and requirements of the two systems.

      More generally, stuff like this…

      Just scapegoat the human and let the broader system get off scot-free. Don’t ask if the equipment was getting out of date, or if the training program is underfunded, or if the official processes were disconnected from the groundlevel reality. When all defects are the fault of the human, then we don’t have any tools to address defects besides blaming humans. And that’s completely inadequate for our purposes.

      …ignores that systems are set up by and signed off by humans. None of this stuff just spontaneously arises. Depending on random npm packages with no security guarantees or code review isn’t a system problem, it’s the fault with the person that added that dependency or the person that signed off on allowing random unreviewed dependencies.

      Making things “system problems” is just a way to shift blame from individuals to everyone because “you can’t fire EVERYONE”. But actually the people that sign off on these systems, the people that get paid HUGE amounts of money on the basis that they’re supposed to be held responsible when something goes wrong, but who somehow never are? The blame is on them. If you tell developers not to review the contents of NPM packages because you have a tight schedule and don’t have time for it then YOU AND ONLY YOU are responsible for the consequences of that decision. Not the system. Not the collective.

      If you don’t have time or resources to do your job professionally then you need to take a stand and tell you manager so. In writing. “I can’t be sure of the quality of this if you do not give me time to either review the dependencies deeply or write that code myself”. Be a professional.

      The only way the public is going to get software that works and is safe and isn’t riddled with security holes is if programmers start creating an expectation of professionalism. Nobody says ‘oh well this bridge doesn’t REALLY matter so we can just cut corners’. Software might not always have the same potentially bad consequences for physical safety (although often, increasingly often, does) but has serious consequences for emotional, psychological and privacy safety. What’s more people should be able to expect software to work, to always work, to work properly and accurately and safely. If that costs more then it costs more. I’m sure unsafe bridges would be cheap too.

      1. 20

        Having, uh, worked on, well, not bridges before, but some physical objects, I really, really hate it when reasoning by analogy is brought up, in the form of bridges or cars or whatever.

        First of all, everybody asks which corners can be cut, all the time. An unsettling amount of regulation is in place precisely in order to prevent people from cutting the wrong corners, and oftentimes precisely because someone did cut the wrong corner in the hope of getting slightly richer, but they ended up killing someone in the process. (Yay for insurance, though – they really did get slightly richer most of the time).

        Second, in other engineering disciplines (electrical, in my case, although I’ve always been an intruder in that field and haven’t really done any EE work in years now) the fact that “the system” sometimes doesn’t work is perfectly understood. Yes, it’s usually one person (or a group of persons) that makes the mistake – the efficient cause of the mistake, if you will – but in a well-lead team, that rarely happens in isolation from others, and purely as a personal failure. Most bad decisions I’ve seen happen at the intersection of:

        • Incomplete understanding of a problem’s implications among management ranks
        • Incomplete understanding of requirements, deployment constraints, usage patterns etc. among engineers
        • Lack of patience among management ranks (“I don’t need all these details, just give me, like, an executive summary” – it’s not like a complex problem magically becomes simple once the gremlins realize you’re busy)
        • Impedance mismatch at the middle-management layer

        In these cases, many mistakes really do end up being “the system’s” fault – in that, although someone’s wrong calculations or wrong prioritization really did end up causing the screw-up, no one, not even that person, can really be blamed for what they did.

        “Defensive” responses to this are absolutely common (and this article is one of them, too), at both ends of the corporate ranks. Upper management & co. insists that personal responsibility is key and that if you think something is wrong, you should speak up and improve the process – gleefully ignoring the fact that doing so usually gets you ignored (see points 1 and 3 above), and sometimes fired if you go over middle management’s head. Middle management and engineers will insist that fault rests solely with the stewards of the system – gleefully ignoring the fact that, indeed, no system is completely impervious to someone doing their jobs sloppily.

        Of course, the people at the bottom of the hierarchy don’t have any subordinates to shift the blame to, which is why the former reaction ends up getting more street cred, but that’s a whole other story…

        1. 5

          Ok, it’s the fault of individual developers. What then? What are you offering as a solution to that from a practical point of view? Empirically, appeals for professionalism and accountability clearly don’t work.

          Besides, bridges also seem to collapse with some regularity in places like the US and Italy, so I don’t know if that’s a useful counterexample and I don’t know if appealing to professionalism is going to solve that problem either.

          1. 4

            Depending on random npm packages with no security guarantees or code review isn’t a system problem, it’s the fault with the person that added that dependency or the person that signed off on allowing random unreviewed dependencies.

            Did you … did you read the thing in the link?

            1. 2

              programmers start creating an expectation of professionalism

              I observe a power imbalance here. Most programmers I know are very quick to shift the responsibility away: “It is not my job to prioritize” and “I need to ask my manager”. On the other hand, managers practically never do that: “I will clarify that” or “I will follow up on that”. It does not mean that managers actually solve more issues than programmers but it creates a perception that the managers are the doers and programmers are the whiners. If programmers never take the responsibility, they will never get the power either.

              Disclaimer: Of course, there comes a risk with taking responsibility. In some circumstances it might even be a trap.

              1. 1

                This is both a completely true and also effectively useless. As a purely practical consideration @hwayne’s approach to analysis and addressing issues has a higher likelihood of success than blaming human error.

              1. 2

                Looks nice, would be cool if it was more mobile friendly, though? I quick tested with the responsive design mode on Firefox, it just gets smaller, which in an actual device is kind of annoying.

                1. 5

                  I adjusted index.html the grid to be responsive.

                  1. 7

                    That’s quicker support than services I paid for, hahahah =P

                1. 3

                  I really like the thrust of this idea, but I’m virtually certain this grid is an abuse of the <aside> tag. Surely there’s a way to achieve the same thing with semantically correct HTML?

                  1. 6

                    Also:

                    • acronym is deprecated.

                    • your index.html lacks its opening html tag.

                    • <h3>Ordered lists</h3> should be an h4.

                    • <label>Number:</label> should be an h4, and the subsequent input elements should be labelled:

                      <h4>Number</h4>
                      <label style="display:block"> <!-- ideally avoid <br/> -->
                        <input type="radio" name="radioSet" value="uno"/>
                        Uno
                      </label>
                      <label> ...
                      
                    1. 4

                      You must be fun at code reviews.

                      Just kidding, I actually admire people which a keen eye for detail. Carefully examining other’s people code can be as helpful if done with kindness as it can be bad if done condescendingly or arrogantly, and you didn’t really do it in either of the later bad ways =)

                      1. 3

                        Thanks! I will fix that.

                      2. 1

                        The project is build around the premise of default CSS starter kit. Instead of using CSS reset you use something like this to set the default style. You will probably need classes to make more complex grid. The grid logic is taken from: https://github.com/vladocar/infinity-css-grid .

                      1. 6

                        I don’t like to blame Java for all woes in programming, but it’s really hard not to blame this one on it. Verbs should generally not be entities. The naming gives you the clue: if you’re having to invent nouns from verbs to name your classes, then perhaps they should not really be classes.

                        Now, in less restrictive languages, this is a simple fix: create a function instead. But then Java came along, decided that everything should be a class (which is actually the problem. Everything being an object is fine, and actually has nice benefits, like easily enabling functions as parameters, as long as you can have function objects =P), and poisoned the minds of so many young people.

                        1. 1

                          Did anyone else read “Bad motherfuckers: how to recognize and avoid them”?

                          1. 2

                            Just check their wallet.

                          1. 3

                            I do it in Python occasionally.

                            1. 1

                              Have been bitten by this in python when a library I imported redefined a global object. It was not a fun debugging session.

                              1. 2

                                Yeah, can imagine. I usually only use it under extreme circumstances, and with a context manager, to minimize the impact. Kind of similar to what patchy does (although patchy is even more extreme and less fun to debug)

                              2. 1

                                Python only allow it to a certain extent. You can monkeypatch any module, even stdlib ones, but you can’t monkeypatch the builtin types or instances of builtin types. modules from C extensions are not monkeypatcheable as well, I think?

                              1. 13

                                The below essay is Frankfurtian bullshit. I wrote it in the same style as https://shouldiusethreads.com/, because I believe that it is also bullshit of the same type; all of the statements are true, or at least very very hard to falsify, but it’s written primarily to offend, not based on its actual truth or falsehood.


                                Should I use the filesystem?

                                No

                                Some people would have you believe that NULL is the worst mistake in computer science. They make a good case for it, but they’re dead wrong. Shared mutable state is the worst mistake in the history of computers. Ask anyone who has debugged both a segfault and a race condition. They will assure you that the latter is 10-100× more difficult to solve.

                                Look at a list of files that your system has touched recently. How many of them have actually been manipulated by more than one program? I’m mostly thinking of the hidden ones, like .bash_history and the .dvdcss directory. Outside of highly-visible but comparatively-rare circumstances, the filesystem is an excessively general abstraction for the “silent majority” of its use. The poorly-defined concept of filesystem atomicity result in a system that simultaneously has both excessive synchronization and not enough synchronization for the purposes of the same application (PostgreSQL, in the case of those two links). It’s impossible to design an fsync() call that provides full-filesystem atomicity while preventing two applications on the same computer from blocking each other, because fsync() is fundamentally a filesystem-global lock.

                                What’s worse is that the default behaviour is to allow limitless race conditions. A process has access to the same filesystem space as other processes in your machine, and can do whatever they want with it (as long as they’re running as the same user, which is the default behaviour when you launch processes). You have to take extreme pains to avoid accidentally doing the wrong thing. You’ll probably mess it up, and the symptoms will show up in an unrelated part of the multiprocess system 10 minutes later. There is no shame in making such mistakes — we’re only human, after all. The shame is in believing ourselves super-human, and reaching for a tool that we don’t need, in full knowledge that we’re likely to shoot ourselves in the foot with it. Nine out of ten times, this tool is the filesystem. If you’re asked to dig a hole and you point a pistol between your feet to get the job done… you should have used a shovel.

                                But, perhaps your problem requires multiple processes to operate on the same data. In that case — you still shouldn’t use the filesystem! Consider using message passing instead. A good model is an overseer program, which organizes the work to be done and aggregates the results, spawning worker programs to run the actual computations. This is not only much more robust, but it scales better, since you could distribute these programs across multiple computers later on*. This approach is more useful, too, because often the user of the program may have a more novel idea about how to distribute work, and if they can invoke the worker processes themselves, they’re empowered to try it out. This is easier to test and debug for the same reasons. And, in addition to all of these benefits, you get brand new virtual memory scratch space which is much more difficult to fuck up.

                                What’s worse is that, while there has been a great deal of attention paid to memory-safety, filesystem-safety in programming languages is essentially nonexistent**. Python, Java, Rust, even Haskell expose almost identical APIs for manipulating files, and that interface is terrible. Database designers since before I was born have known that user data and query code should be connected using prepared statements only, but that’s not how you manipulate paths on the filesystem on any of the major operating systems. If you want to connect user input that might potentially contain a forward slash, the best thing that language abstractions might provide is a function that will error out in the presence of “forbidden characters.” At least HTML has standardized escape sequences; how is it acceptable that, in 2020, we can’t have forward slashes in file names? You want to scrape the IMDB and use it to populate file names? Sorry; those episode titles have forward slashes in them, so you’re going to have to invent some arbitrary, non-standardized character substitution and hope it doesn’t result in a naming conflict. “Don’t fuck with user input” is a pretty low standard for API design, and the filesystem paths fail it.

                                Don’t use the filesystem! It’s a trap!

                                * NFS is not a solution. Because it’s designed to be transparent to the application, it can’t be both reliable and fast, because the application doesn’t give it enough information to know whether an operation should be retried or failed out if something goes wrong.

                                ** The closest you can get is sqlite, which is great, but it can’t really fix the pessimistic synchronization, and it doesn’t work very well with existing tooling like git.

                                1. 6

                                  Honestly, this mostly makes sense to me? The fact that we don’t have any better solutions to the use cases filesystems address doesn’t mean that a global shared filesystem is a good idea to begin with. In fact, if you look at more recent OSs that cared less about retro compatibility, like the mobile OSs, they all have some sort of per-app scope for filesystem, and in iOS specifically it’s something very abstracted over.

                                  I mean, if you were designing a OS today, without any care for any retro compatibility, without any need to support any sort of posix API, would you really provide files and a filesystem as an abstraction for storage? I don’t think I would.

                                  1. 1

                                    I can’t decide.

                                    On the one hand it feels pretty archaic to just store labeled blobs of bits vaguely sorted in a tree and maybe with an extension as a rough type indicator. How are we going to explain this to the kids swiping on the TV? ;) And that in times where we have cloud everything and our phones have share buttons to avoid dealing with files! I am totally on-board with this accidentally true rant.

                                    On the other hand pretty much every order I need I can create with the wonderful filesystems we have and I prefer that to dealing with having my files scattered over sharepoints and onedrives and google docs and whatnots.

                                    1. 2

                                      How are we going to explain this to the kids swiping on the TV?

                                      The same way it was explained to me? They can learn like anyone else, it’s not unlearnable.

                                      1. 2

                                        Your second alternative is not the only other option to filesystems though. It might seem like the only feasible one, but I’m not constraining myself with such minor concerns as “is this even remotely viable in the real world at all”.

                                        What if we could have one (or a couple) rich, programmable, safe, and portable interface that every program could (and did) use? What if you could have even more flexibility (or rigidity) at your fingertips to organize your data, in pretty much any device?

                                    2. 1

                                      You nailed it! I would just add a few solutions like “use INT 13h directly” or “print your data on paper”.

                                    1. 1

                                      This seems all very nice indeed, but every time I embark on some customization journey I eventually give up, for one reason: reproducibility.

                                      Do you use more than one computer? Now you have to replicate all of that in all of them. Do you use more than one OS? Same. Reinstall/update the OS? Either do everything again or have to fix it because the update broke it. Need to do something in someone else’s computer? It’s gonna be a pain because your muscle memory will be all over the place.

                                      None of this is unsolvable, or even arguably that hard. But I don’t usually have the patience to do it, so, I tend to stick to vanilla as much as possible. If something is really annoying I might change a config or add a program to help (clipboard manager is a thing I can’t live without, for instance).

                                      1. -2

                                        I appreciate the time that you took to write this post. I know that the Python community, including myself, can be aggressive and unpleasant in online discussions.

                                        The PyPy project, for example, is currently dependent on a Python 2 interpreter to bootstrap and so will be maintaining their own either for as long as PyPy exists, or for as long as it takes to migrate to bootstrapping on Python 3 (which they seem to think is either not feasible, or not something they want to do).

                                        This quote is emblematic of the contempt that you display towards Python users. You imply with your words here that porting RPython to Python 3 is “feasible”, but that ignores the cost of labor. Multiple projects have found that Python 3 porting is too expensive to tackle without dedicated funding and support, including not just Django and PyPy, but also Twisted and entire distros like Ubuntu and Gentoo; if those projects cannot find funding then they do not do the porting.

                                        I don’t understand your denigration of interpreters. Producing and maintaining an interpreter is non-trivial, a language community should strive to have a heterogeneous collection of interpreters and compilers besides the reference implementation, and the big list of Python 3 libraries pledging to drop Python 2.7 is missing not just PyPy, but also Jython and IronPython. While some implementations, like MicroPython have switched exclusively to Python 3, others have not.

                                        To paraphrase somebody else’s insight from an IRC discussion, perhaps, as with Perl and Raku, it is the case that Python’s community became too ideologically burdened, and the fork is the result of an ideology operating without constraints. After all, my observations are only weighty because of the inertia of code running on computers, and perhaps that is the only argument in favor of Python 2: That it is the way that things are being done, regardless of the loudness of the claims of Python 3 proponents. I certainly am not here to praise Python 2 either, but to bury it; in fact, were the language still in one piece, I would only have to dig one tomb and write one eulogy. I need to credit glyph for being the one to first paraphrase Marc Antony, over a decade ago at PyCon; I was so slow to understand what everybody else was trying to point out.

                                        Though there is one thing I think gets overlooked a lot: usually, the anti-Python-3 argument is presented as the desire of a particular company, or project, or person, to stand still and buck the trend of the world to be ever-changing.

                                        But really they’re asking for the inverse of that. Rather than being a fixed point in a constantly-changing world, what they really seem to want is to be the only ones still moving in a world that has become static around them.

                                        I am not asking for either of these positions. First, let me dispel this idea as you’ve worded it. I understand that a language is not fixed in stone, and that it will constantly change and evolve with usage. Indeed, I believe that languages are expressive media, and that the main purpose of language is to send messages and communicate, but since our understanding of the world changes over time, our languages must change as well. (Also, explicitly, I am not wearing any corporate hats. I have many commit bits, but the only relevant plan I currently have is to remove Python entirely from all active codebases, incrementally and immediately.)

                                        My argument, in ten words or less, is: Okay, yes, but must we have changed “2” to “3”? The massive amount of backporting done so far has revealed that no, most of the new features in Python 3 could have been put into Python 2. People had already, as you pointed out, survived having to change with to be reserved and then a keyword, and that change was far more intrusive than anything in the 2-to-3 changelist. Meanwhile, the modules that ought to have been killed off directly, most notably asyncore, were left in Python 3 despite being effectively unmaintained and long since rotting.

                                        1. 16

                                          the only relevant plan I currently have is to remove Python entirely from all active codebases, incrementally and immediately

                                          Surely it would be less work to migrate your Python code to Python 3 than to rewrite it in an entirely different language.

                                          1. 12

                                            He’s having a tantrum.

                                            And I can say that, having overseen the 2 to 3 conversion for a large adtech codebase and maintained 2/3 compatible libraries. I did that because it was both challenging and required really creative, often perverse, solutions. And this was with my mental model very carefully keeping track of string/binary boundaries. It wasn’t easy and people who balanced both interpreters and didn’t collapse under the devilry of duplicate code (and subsequent bug duplication) have my admiration.

                                            If I could move a stodgy adtech company to 3 for active projects, then he can as well.

                                            I’m not a super coder, just someone who adds asserts and routinely retests her assumptions. If you’re not on 3, it’s either a big ball of mud that should’ve been incrementally rewritten/rearchitected (thus exposing bad design) or you expected an ecosystem to stay in stasis forever. If you’re the latter, compile from source and lockdown your environments more than ever cause you’re on your own.

                                            Already I’ve witnessed removal of python 2 from distributions! It’s getting killed regardless of how anyone feels. Digging ones heels in is just silly.

                                            1. 0

                                              You should work on your mind-reading. My complaint is mostly that, for over a decade, PyPy developers have been consistently marginalized by the PSF. This forced deprecation of Python 2 is merely one more brick in the wall. While I am fortunate to be independently wealthy, and I also barely contribute to PyPy, the typical PyPy developer is a hard-working academic who is trying to dramatically speed up the Python ecosystem, and the PSF’s continued snub of their work is disappointing, especially given their continued endorsement of corporate-sponsored JITs hacked on top of CPython, like Unladen Swallow, Pyston, and Pyjion.

                                              I’ve overseen 2-to-3 conversions as well. I’ve seen companies that can do it fine, companies that are ready with 2to3, and companies that are totally hopeless. I’ve seen companies that are so behind on security patches that they cannot upgrade away from Python 2, and companies that greenfield with Python 3. It is hard but possible. The main sticking point, which folks seem to constantly paper over, is that Python 2 and Python 3 are fundamentally different langauges, with different grammar and semantics. Indeed, PSF representatives have explicitly asked me to not talk about Python 2 and Python 3 as if they are different, and when I’ve asked followup questions, they’ve indicated that it is because treating the two languages as different undermines the porting effort. This is how the system works.

                                              It is funny that you mention “compile from source;” I use Nix, and so of course there will be CPython and PyPy for Python 2.7 available for a very long time via nixpkgs. Again, I am independently configured, and I am not a part of any core development team. But I am not “on my own;” I am one of a community which maintains the ports tree.

                                              1. 3

                                                Python 2 and Python 3 are fundamentally different langauges, with different grammar and semantics

                                                No they’re not. The existence of 2/3 compatible packages utterly shits on your point.

                                                The only “breaking” change that you seem to be whining about as soooo different is the splitting of a primitive type. That’s it. Splitting a fucking type into two. That’s not a fundamental language difference and you know it. You’ve made backwards incompatible changes in your code bases and most likely have split classes and refactored. Tell me, was your program utterly grammatically different? Probably not.

                                                You didn’t whine about long/int merging because it’s a trivial alias to add.

                                                But splitting a type is always a lot more contentious and difficult (probably verging on the biggest type of breaking change possible).

                                                Removing “print” so that it could be substituted with a function hardly counts as a massive grammer change. And the addition of coroutines is strictly an “add on”.

                                                I’m not going to even bother with your “mother loved you best” vis a vis PyPy. Jython has been effectively dead for years (unless they finally got invokedynamic support to actually get usable speeds - I last checked two years ago, so sue me).

                                                I don’t know about the others as they’ve been more like toys than have capability of supporting the ecosystem as-is. You can’t just demand equal buy in without supporting the ecosystem as-is - PyPy literally had to make c-extensions work after claiming for years it was impossible to do (which was absurd - I remember reading one of the JVM Ruby interpreters ended up supporting c-extensions through parsing the c-extension source code into compatible libraries - if ruby could do it, so can python).

                                                1. 6

                                                  Please lower the temperature on your posts. You don’t need to curse and make insulting characterizations of his post to make your points.

                                                  1. 10

                                                    Your points withstanding (I should be less overtly crude), the GP’s routine use of “contempt” as an accusative is nothing short of insulting. He paints himself as the victim of the PSF and anyone who agrees with them as doing their bidding. It is extremely unkind and quite conversation terminating.

                                                    I get that you feel you’re being fair, but it comes across as uneven enforcement amidst the unfair characterizations and portrayal of an anti-Python 2 conspiracy, when in reality it’s an interpreter hide-bound by Guido’s rules to keep it as simple as possible (thus hamstringing backwards compatibility like seen in the JVM), a hold over from the early anemic days of Python and something that’ll never change.

                                                  2. 2

                                                    The existence of polyglot libraries does not “shit[] on my point;” it merely shows how far people can be convinced to go in pursuit of a satisfying end-user experience. The existence of libraries which compile under both a C compiler and also a C++, Objective-C, Zig, or etc. compiler does not invalidate the point that the languages are different, regardless of whether the program texts are identical in those cases.

                                                    Here are two short interactions with two CPython versions. I contend that they implement distinct grammars and semantics from each other.

                                                    >>> {1:2,3:4}.items()
                                                    [(1, 2), (3, 4)]
                                                    >>> {1:2,3:4}.iteritems()
                                                    <dictionary-itemiterator object at 0x7f704a55f170>
                                                    >>> nonlocal = 42
                                                    >>> class Foo: pass
                                                    ...
                                                    >>> Foo.__mro__
                                                    Traceback (most recent call last):
                                                      File "<stdin>", line 1, in <module>
                                                    AttributeError: class Foo has no attribute '__mro__'
                                                    

                                                    And the other one.

                                                    >>> {1:2,3:4}.items()
                                                    dict_items([(1, 2), (3, 4)])
                                                    >>> {1:2,3:4}.iteritems()
                                                    Traceback (most recent call last):
                                                      File "<stdin>", line 1, in <module>
                                                    AttributeError: 'dict' object has no attribute 'iteritems'
                                                    >>> nonlocal = 42
                                                      File "<stdin>", line 1
                                                        nonlocal = 42
                                                                 ^
                                                    SyntaxError: invalid syntax
                                                    >>> class Foo: pass
                                                    ...
                                                    >>> Foo.__mro__
                                                    (<class '__main__.Foo'>, <class 'object'>)
                                                    

                                                    I am not sure why you characterize my points as “whining” and then agree with them by substantiating the needful evidence. It seems that you are angry with me because I have points, not because I don’t have points.

                                                    By using a broad brush to label various Python projects as “toys” or “dead”, you only emphasize the culture of contempt which I highlighted in my thread-starting comment. Moreover, you’ve said the quiet part out loud: The reason for this contemptuousness is a tribal belief in the power of C extensions over plain old Python code. What’s remarkable to me is that this belief isn’t actually in the service of speed or compatibility; the PyPy project produced not just PyPy, but also CFFI, in order to put those claims to rest. Instead, the belief seems to be simply one of majority rule: Because CPython is the popular standard, CPython must remain the popular standard. To quote from The Power and The Glory:

                                                    And though the hard times are really due to me, it is still in me to quell all this unrest

                                                    Things must stay, there must be no change; anyway, time to rearrange

                                                    You must believe me, that there’s been no such betrayal, all that I’ve done, I’ve done for you

                                                    Things must stay, there must be no change; anyway, time to rearrange

                                                    There must be a reason why plans have turned around! Not only in me, it must be in you too

                                                    Things must stay, there must be no change; anyway, time to rearrange

                                                    1. 3

                                                      Uh, no. With the exception of nonlocal as a keyword (which btw is no different when with as a keyword was introduced), all the examples cited are properties on data objects, which is categorically not “grammar”.

                                                      Grammar pertains to parsing the language into an syntactically valid AST from which a bytecode suitable for the VM can run on. The only incompatible grammatical changes I can think of off the top of my head are:

                                                      • with (python 2.4)
                                                      • print to print() (Python 3)
                                                      • nonlocal (Python 3) (similar to global except restricted to scopes excluding global)

                                                      That’s it.

                                                      Write a proxy for the builtins and you can support older method returns without a problem. That’s why six was did it’s job so well.

                                                      By using a broad brush to label various Python projects as “toys” or “dead”

                                                      Does it support the ecosystem? If not, then not very comparable with CPython in now, isn’t it?

                                                      As to dead, Jython still doesn’t do invokedynamic and that’s pretty critical to decent performance. Python is well known to be pretty slow. Jython is even slower. That heavily constrains use case to “we have to run a Python script on the JVM” and that’s about it. And it’s also stuck on 2. So it’s left behind in time to a shrinking ecosystem of pure python 2 modules. Talk about useful 🙄

                                                      Moreover, you’ve said the quiet part out loud: The reason for this contemptuousness is a tribal belief in the power of C extensions over plain old Python code

                                                      Actually, I didn’t. You made that up all by yourself honey.

                                                      I only cited the support of C-Extensions as part of recognizing the audience demand for being able to use PyPy in a broader case than just pure Python and rare/underused one shot ports of C libraries and C-compatible library exports (did several for previous employers) that typically used C-extensions/Cython.

                                                      You’re so wrapped up in inventing enemies that heap contempt on you, but it’s just fellow engineers raising their eyebrows at someone being overly dramatic. Lol contempt. 😂😂😂

                                                      1. 11

                                                        I need to go update it for 3.8 final and things coming in 3.9, but I maintain a list of ways to “break” Python, geared at library authors who want to trigger syntax or import errors as quickly as possible when someone tries to use one of their libraries on an unsupported Python version.

                                                        It also serves as a sort of tour of changes to Python syntax over the past decade.

                                                        1. 2

                                                          I like this! It shows the grammatical additions (SyntaxError raising ones) fairly well!

                                              2. 4

                                                I don’t have any real opinion on python2/3 myself. However, if the language you use forces you to make changes which you disagree with, then it is not unreasonable to at least consider the other options out there. Maybe it’s simple enough for them to port from python 2 to 3, but will they then have to port from 3 to 4 in the future? Maybe porting from 2 to 3 isn’t a massive deal, but they weren’t super happy with python for other reasons, and it was the straw that broke the camel’s back.

                                                1. 7

                                                  It is more or less twenty years since discussion started about possible breaking changes that eventually became Python3 and 14 years since “official” start. We are not even at the first step for Python4. Based on how often most people change jobs and projects in this industry, it is very unlikely they will need to port to 4 or even worry about what that will entail for a completely imaginary version of the language. They are likely to go through far more impactful transitions before then.

                                                  These days I mostly write Python 3 and on occasion still Python 2. I find the notion that they are different languages absurd.

                                                2. 2

                                                  Yes, and it would also be less work to use Python 3 than to write an entirely new language. However, we have reasons, and so we produced Monte, a modern flavor of E. We were already leaving Python, and we were already rewriting our Python-based virtual machine in Monte.

                                                3. 26

                                                  This quote is emblematic of the contempt that you display towards Python users. You imply with your words here that porting RPython to Python 3 is “feasible”, but that ignores the cost of labor. Multiple projects have found that Python 3 porting is too expensive to tackle without dedicated funding and support, including not just Django and PyPy, but also Twisted and entire distros like Ubuntu and Gentoo; if those projects cannot find funding then they do not do the porting.

                                                  I’d like to think I know a thing or two about large projects porting, and about funding situations.

                                                  (for the uninitiated: I got my commit bit on Django in 2007, was its release manager from then up to about 2014, served multiple terms on the framework’s technical board, and am currently in my fifth year on the board of the DSF, the nonprofit foundation set up to support Django and its community, including by fundraising for, and then funding, its development)

                                                  Also, hi. I am a Python user, and I’d seriously dispute your assertion that I have “contempt” for Python users.

                                                  I don’t understand your denigration of interpreters. Producing and maintaining an interpreter is non-trivial, a language community should strive to have a heterogeneous collection of interpreters and compilers besides the reference implementation, and the big list of Python 3 libraries pledging to drop Python 2.7 is missing not just PyPy, but also Jython and IronPython. While some implementations, like MicroPython have switched exclusively to Python 3, others have not.

                                                  I don’t understand why you interpret it as “denigration”. You’ve repeatedly indicated your dislike for the Python core team’s and the PSF’s messaging that Python 2 is no longer supported, and you’ve argued that Python 2 is supported for as long as alternative Python 2 interpreters are supported. My point is that while a supported interpreter is a necessary thing for calling Python 2 “supported”, it is not sufficient. When people say they want to keep using Python 2, they mean they also want to keep using all their favorite packages and libraries, and many of those simply do not run on Python 2 anymore. And as far as I know, of non-CPython implementations still supporting Python-2-the-language, only PyPy has ever been robust enough to support some of the really important packages like NumPy/SciPy.

                                                  Anyway, this is why I bring up the question of whether you’re willing to commit to also maintaining a complete Python 2 ecosystem including the big third-party packages. The Linux distros who’ll be maintaining Python 2 interpreters for years to come have committed to that, but as far as I can tell nobody else has. And without the ecosystem I don’t think you have a sufficient “Python 2” and as I’ve told you directly and also mentioned in the post, I think you create unnecessary and unwanted burdens on those packages’ maintainers if you insist on saying “Python 2 is still supported”.

                                                  The massive amount of backporting done so far has revealed that no, most of the new features in Python 3 could have been put into Python 2.

                                                  OK, but now make a Python 2 that cleanly implements the string change and the reorganization of the builtins and the standard library and doesn’t create a massive backwards-compatibility gap to older Python 2 versions. I don’t think that’s possible, and I do think that at least the string change is an absolutely necessary thing for Python’s future.

                                                  1. -3

                                                    The Python Packaging Authority (PyPA) is looking to get rid of Python 2 support, at which point I suppose you’ll shift even more of the burden to the remaining Python 2 users somehow, despite the singular package index still being squarely under PSF control.

                                                    The rest of your post fails to address my point. If you can’t stop talking past me, then please don’t bother. You can’t even look at a Python 2 user without the insinuation that they must be responsible for the entirety of their ecosystem.

                                                    Frankly, I think that you show your hand when you say “really important packages like NumPy/SciPy.” That’s the direction that you want Python to go in.

                                                    1. 14

                                                      at which point I suppose you’ll shift even more of the burden to the remaining Python 2 users somehow

                                                      Well, I’m not trying to be mean about this, but: this is what the “entitlement” section of the post was about. The PyPA folks don’t owe you a Python-2-compatible packaging toolchain. The Django team doesn’t owe you a Python-2-compatible version of Django. And so on.

                                                      If those teams have decided they only want to support Python 3, they have the right to do that! And if you want to keep using those tools and libraries and packages with Python 2, you can. You just have to choose between:

                                                      • Use an older version of the project with no more bugfixes or security support, or
                                                      • Form a group of like-minded people to do your own bugfixes and security support

                                                      Way way back in the early days of Django when the “magic-removal” effort rewrote the ORM (between Django 0.91 and 0.95), creating a big backwards-incompatible jump for projects to get over, I and some other people took the second option: even after the core Django team wasn’t supporting Django 0.91 anymore, we maintained a Django 0.91 branch for a while in order to gain more time to migrate. It’s open source and you can do that.

                                                      You can’t even look at a Python 2 user without the insinuation that they must be responsible for the entirety of their ecosystem.

                                                      I could just as easily flip this around and say you can’t look at someone who develops on Python 3 without insinuating they owe you their time and effort to also maintain a Python 2 version of whatever they build.

                                                      When you say “Python 2 is still supported”, you seem content to mean it in the specific narrow sense of “there is at least one group still maintaining a Python 2 interpreter”. What I’ve told you many times now is that when people hear “Python 2 is still supported”, what they will expect that to mean is a wider sense of popular third-party packages still maintaining Python 2 compatibility. I have concerns about that, because I maintain third-party packages and I’d rather not have a stream of people showing up in my bug trackers and getting angry because they heard “Python 2 is still supported” and thought that would mean I’ll maintain packages for them to use on Python 2.

                                                      Frankly, I think that you show your hand when you say “really important packages like NumPy/SciPy.” That’s the direction that you want Python to go in.

                                                      NumPy and SciPy are important packages. I don’t personally use them – as I’ve mentioned now multiple times in these discussions with you, I’m a Django guy, and my main use case for Python is as a backend web development lanaguage – but I know that there’s a large segment of the Python community who rely heavily on the numeric/scientific stack to do their work. I don’t see how anyone could look at the Python community and not come to that conclusion. So regardless of whether I use them or not, NumPy and SciPy are important packages. Just as Jupyter (née IPython) notebooks are important, even though I don’t personally use them. Just as the ML/AI packages are important even though I don’t use them. Just as Flask and SQLAlchemy are important packages, even though I don’t use them. Python’s continued success as a language comes from the large community of people using it for different things. The fact that there are large numbers of people using Python for not-my-use-case with not-the-libraries-I-use is a really good thing!

                                                      Anyway. You’re angry at what you see as the marginalization of alternative Python implementations, and especially the one you personally are most involved with (PyPy). That’s very clear (in fact, you’ve even said that pretty explicitly). But you’re so angry about it that you’re lashing out at random people on the internet, and either ignoring or misinterpreting what they say.

                                                      And on one level, I get it. The alternative implementation story has been rough! Django used to support IronPython and Jython, for example, and we treated it as a release blocker if we had compatibility issues with them. But at a certain point they started falling further and further behind CPython, and eventually it was clear that it would be a long time before they caught up, if ever. That’s sad – it used to be a great way to sneak Django into a Java or Windows shop – but I don’t believe Django has the resources to keep supporting such widely-diverged platforms long-term. If and when Jython and/or IronPython catch up to CPython (and both of them seem to have Python 3 compatibility as a goal), I’d be happy to push for Django to maintain compatibility with them again.

                                                      But on another level, you’re not really helping your case when you treat people as your enemy for no longer supporting Python 2 in their packages. If I didn’t already have a long history of knowing other PyPy people, for example, I’d be coming away with a pretty negative view of the project from my interactions with you.

                                                  2. 10

                                                    also Twisted

                                                    This document is obsolete. Twisted has been ported to Python 3 (jettisonning a few unmaintained modules in the process). The next release will require Python 3, likely 3.5+.

                                                    1. 2

                                                      Twisted did take about 15 million years to port, though, so, it does speak to how difficult porting can be. They did start a long time ago, and also have pretty much finished a while ago as well, so, anyone who hasn’t started should’ve kinda seen this (the end-of-life for Python 2) coming.

                                                      1. 7

                                                        Yeah, it was a huge project, and particularly difficult because Twisted is exactly the sort of code that is most difficult to port: it sits at the divide between bytes and str. Something like a Django app is much easier to port — both because of the solid groundwork laid by Django itself, and also because the bulk of a web app’s logic is only concerned with text.

                                                        I do think that the OP undersells how unevenly the pain of the Python 3 transition has been distributed. It has been particularly rough for library maintainers, who often want to retain compatibility for a long time to enable their users to migrate. The end result has been a slow and painful transition with the corresponding toll on the Python community.

                                                        I would really like to see a blameless post-mortem of the Python 3 transition. Surely we can learn some lessons from this process that apply to development going forward.

                                                        1. 7

                                                          I do think that the OP undersells how unevenly the pain of the Python 3 transition has been distributed. It has been particularly rough for library maintainers, who often want to retain compatibility for a long time to enable their users to migrate.

                                                          Assuming by “OP” you mean me (author of the blog post this whole thread is about), I don’t think I’m “underselling”. I was just trying to write about a bunch of different thoughts and perspectives that I think either haven’t been covered, or haven’t been covered enough, in the copious literature on the Python 2/3 transition.

                                                          Something like a Django app is much easier to port — both because of the solid groundwork laid by Django itself

                                                          This is sort of what I was getting at, though. I remember the days of Django’s “unicode branch” and how much work it took to get Django to a point where Django apps mostly didn’t have to worry about that stuff. As the saying goes, a lot of effort went into making that look effortless, and Django’s early experiences strongly influenced my later feelings about the right way to handle strings.

                                                          1. 4

                                                            Assuming by “OP” you mean me (author of the blog post this whole thread is about), I don’t think I’m “underselling”.

                                                            Perhaps not the best word choice on my part.

                                                            I do think that the experience of making the transition varies a lot, and that may explain some of the heat in this thread and the discussion in general. We’re all frustrated by how things have dragged out.

                                                            As the saying goes, a lot of effort went into making that look effortless, and Django’s early experiences strongly influenced my later feelings about the right way to handle strings.

                                                            Yes, exactly! A post-mortem should focus on what Python could have done to make the transition work this way for everyone.

                                                      2. 2

                                                        Sure. The links I used fail to actually tell the story that I wanted to tell. There was a point in time at which most of Twisted was Python 2 and 2to3 wasn’t capable of making up the difference. At that time, people often asked for Python 3 support. However, Python 3 support was only happening through paid work, and community members were not spending time on the porting effort. Eventually, as you say, many modules were simply removed entirely rather than being ported.

                                                        This is the process that I wanted to emphasize as happening repeatedly all over the Python ecosystem. Every library maintainer was asked to put in massive amount of effort, to the point where external funding was considered necessary to get things done. But this effort was only worthwhile because Python 3 supposedly was better than Python 2.

                                                      3. 7

                                                        My argument, in ten words or less, is: Okay, yes, but must we have changed “2” to “3”?

                                                        I think that misses the point the author is trying to make. He argues that the maintainers of python ARE moving from 2 to 3. What we/you must do in this context is either assume the maintenance burden of a forked python 2 or migrate to python 3. You’re welcome to do neither as well, but of course any bugs related to your ecosystem will go unpatched.

                                                        1. 4

                                                          People had already, as you pointed out, survived having to change with to be reserved and then a keyword, and that change was far more intrusive than anything in the 2-to-3 changelist.

                                                          Sorry but just to clarify: do you genuinely think that a keyword becoming reserved is a more intrusive change than switching to Unicode strings and explicit conversion to and from Unicode strings and byte sequences?

                                                          1. 0

                                                            As you’ve phrased it, no. However, what you’ve described is not what actually happened. Python 2 supports the Unicode sandwich technique just as well as Python 3, and has the unicode type. I would rephrase your question as, “Do I genuinely think that a keyword becoming reserved is a more intrusive change than renaming unicode and str to str and bytes respectively?” And the answer is yes, I do.

                                                            1. 11

                                                              From your link:

                                                              So Python 2’s pain is deferred: you think your program is correct, and find out later that it fails with exotic characters.

                                                              With Python 3, your code fails immediately, so even if you are only handling ASCII, you have to explicitly deal with the difference between bytes and unicode.

                                                              I can’t believe that a “security professional” would countenance implicit, latent bugs like this on a fundamental data type. I’ve done the unicode sandwich for years and write implicitly statically typed python as a matter of practice (which JITs nicely in PyPy without having to wonder “will it blend?”).

                                                              Of the several hundred engineers I’ve met and the dozens I’ve worked with daily for years, I can count the number of people who are aware of unicode/str implicit mixing on one hand.

                                                              You’d honestly preserve a constant subtle footgun than explicitly expose the incompatible boundaries so they’re dealt with early on and not left as a landmine to appear years later? Wow

                                                        1. 5

                                                          In the first blog post, linked in the README as well, you say:

                                                          It brings a novel approach to structured concurrency

                                                          But you never really expand on what said approach is, neither on the blog post, nor in the repository?

                                                          1. 6

                                                            s/you/they/g

                                                            AFAIU, the author of the library ≠ the submitter of the story here.

                                                          1. 33

                                                            I think this disadvantage:

                                                            Debugging is harder

                                                            hides an important problem of this approach: the results aren’t reproducible. This is the main reason why you want to start from a clean state – not because it makes it a fair test (real-world usage is rarely fair, after all) but because it guarantees that the entire system state, and thus any subsequent behaviour that depends on it, is under your control.

                                                            (“The entire system state” is, of course, limited by what your test suite includes – I’ve certainly seen test suites that worked on one system and failed on another and it eventually boiled down to different libsomething versions or whatever).

                                                            Otherwise, if a test crashes on Monday’s night automated test run, but not on Tuesday’s night automated test run, you don’t know if that’s because the bug fix you attempted on Tuesday morning really fixed it, or because someone else pushed another feature whose test runs before yours and it just happens to make whatever garbage was in the database go away.

                                                            This also guarantees that, if a test crashes on the Monday night CI run, whoever has to fix in Tuesday morning has to run just that one test, not the whole test suite up to the failing test (since the database isn’t cleared between tests, there’s no reason to assume that whatever you’re left with at the end of the whole run is actually relevant).

                                                            I’ve been bit by this once – I got a bug report about a crashing test, and it was revealed because a previous test didn’t clear the system state (it didn’t clear some packet counters, which caused a whole number of improbable interactions and eventually caused a pointer in an array of pointers (not the same one each time) to point into garbage land. Sometimes. Depending on some counter values, and there were a few hundred counters. A subsequent test would, much later, cause the code to dereference one of those pointers and boom. Fortunately it dereferenced all of them, since it looped over the whole array, so at least it was the same test failing all the time, but that was just a happy accident.)

                                                            On the one hand, that did reveal a nasty bug. On the other hand the test suite had several thousand test cases,it took almost 14 hours to run, and about 8 hours to see the crash. It took me almost two weeks just to get it to happen without waiting for 8 hours, so that I might actually confirm it’s fixed.

                                                            Edit: oh yeah, got sucked into my own story and nearly forgot.

                                                            There is definitely an argument to be made for the fact that you should test your code against bad data. Lots and lots of automated test suites verify functionality by building up a valid system state, and then verifying that the running the system with that state produces the expected (“valid”) outcome. That’s great but getting code to run correctly on correct data isn’t that hard, it’s that pesky invalid data that makes it choke.

                                                            But in order for this to be valuable in terms of debugging and validation, it has to be reproducible bad data. Early in the development process you can probably crash anything by piping /dev/urandom into it, but getting /dev/urandom to produce the same data on two machines is hopefully very difficult.

                                                            There are plenty of ways to systematically produce a useful stream of bad data. At the end of the whole thing, you can certainly top it off with a stress test from /dev/urandom – sure, it’ll be hard to reproduce but it’s better to know that there’s a hidden bug somewhere than to not know it). But it’s not that hard to get bad data that covers most of the frequent failure modes.

                                                            1. -1

                                                              I suppose my response to this is a) if you are having flaky tests perhaps you need to go back to starting each test with a clean state until you resolve it (easily done - as mentioned in the post) and b) perhaps it’s worth looking into whether you can store the state at the start of the test with the aim of aiding reproducibility. Also I’m not sure I would leave dirty data in CI, I’m more thinking about local environments here.

                                                              I think it’s rarely as simple as piping /dev/urandom to your application to generate bad data. If you look at libraries for generative testing you’ll see they have lots of tricks for generation, shrinking, etc to help generate “interesting” random data with low(ish) computational cost. Interesting tidbit: I use generative testing libraries a lot an occasionally I find that they can go for a while (weeks, and millions of iterations) before finding a counterexample in some cases.

                                                              1. 5

                                                                a) if you are having flaky tests perhaps you need to go back to starting each test with a clean state until you resolve it (easily done - as mentioned in the post)

                                                                So… how can you tell if a test is flaky then? Just because it triggers a crash? That’s what tests are supposed to do.

                                                                and b) perhaps it’s worth looking into whether you can store the state at the start of the test with the aim of aiding reproducibility.

                                                                Isn’t that equivalent to clearing the database between tests and populating it with the (invalid) data that causes the crash? It just takes way longer to get there – plus you get to deal with all sorts of additional problems, like storing and restoring state, which may not be limited to database records.

                                                                1. 4

                                                                  So… how can you tell if a test is flaky then

                                                                  A flaky test can pass and fail when run on the same code.

                                                                  This can be due to state from previous tests (self-poisoning) or ‘hidden inputs’ (eg OS threads getting scheduled/interrupted in a different order, networking etc).

                                                                  1. 3

                                                                    Sure, if a test passes one the first run and it fails on the next, there’s no question that it needs attention. That’s not the problem.

                                                                    Suppose you have the proverbial bug about handling people whose last name somehow evaluates to NULL – so your code crashes when it encounters someone named James Null. You do the fix on your branch, looks okay. You run your test suite on your branch, and it seems to work fine. Nothing crashes anymore, not today, not this week – in fact not for a whole year.

                                                                    Twelve months later the test suite still runs fine. Is that because:

                                                                    a) Your fix was correct and no regression has been introduced, or b) Your fix was accidentally reverted at some point, or it was sloppily ported during a refactoring, or was lost in a complete rewrite of that module – but someone also implemented a “wildcard remove” feature, and their test removes every record in the database that contains the word “James” – including “James Null” – prior to your test?

                                                                    Suppose you did the smart thing, though, and instead of relying on state from previous cases, you’ve now integrated the “James Null” case in your integration test, and you add a “James Null” record as part of the setup. Better yet, to avoid the kind of mishap that resulted in your input being removed just because it had “James” in it, your setup case adds a record with 48 random characters and the word Null. So twelve months later, if your test suite doesn’t crash, is that because:

                                                                    a) You correctly handle the “James Null” case, or b) One of the previous test cases left a trigger function that automatically sanitizes invalid inputs, so “James Null” (ro whatever random input is there) magically gets rewritten to “James \0”, and now there’s no problem anymore, ‘cause that’s just a string like any other?

                                                                    1. 1

                                                                      Yeah, but … how cleaning the database helps with these questions? I’m not seeing it.

                                                            1. 12

                                                              Using a single monitor.

                                                              1. 2

                                                                I’ve always been of the opinion that multiple monitors don’t make sense. I only look at one monitor at a time anyway, and I can set up my window manager to allow me to switch between workspaces on that monitor with less effort than it takes me to yaw my head.

                                                                A well configured single-monitor setup is essentially a multi-monitor setup except all monitors are virtual and materialise right in front of you instead of being stuck to your sides.

                                                                That said, I have been forced to use macOS at work for a while now, for which multiple monitors was more convenient. I switched back to Linux and a proper window manager now while working from home, so we’ll see what happens when I get back to the office.

                                                                1. 2

                                                                  I’m using i3, so it’s basically a single key press for me to switch workspaces, and I’m still super happy to have a dual monitor setup.

                                                                  While it might take the same time to switch workspaces by pressing a key instead of turning your head (I highly doubt that btw) it still makes a difference in terms of cognitive load, because looking around is much more natural than pressing a button. It’s just the way our little monkey brains are wired. With the other workspace you sort of have to be aware that it’s there and actively switch to it (even when you’ve internalized it and built muscle memory). The 2nd monitor is still always in your peripheral vision, you can’t really forget it’s there.

                                                                  I think you can draw an analogy between the dual monitor setup and the “no modes” philosophy of Larry Tesler. With the dual monitors there are no modes and everything is present at all times. With one monitor you have to repeatedly switch to “2nd workspace mode” (and I say that as a huge fan of vim).

                                                                  I have one monitor in portrait mode and the other in landscape. Its really helpful to keep my editor on one monitor and my browser with documentation on the other. Or now for video calls I keep the call on fullscreen on one monitor and use the other to take notes and look up stuff.

                                                                  1. 2

                                                                    There’s a cognitive difference between having two monitors and one, though - they’re in physically different spaces, and the brain pays attention to geometry and positioning.

                                                                    I’m not sure if this particular effect applies to anyone, but in my case, it feels like my brain prefers that I keep two windows in physically different spaces (different monitors) rather than “in the same space”, overlapping and being switched via alt-tab. Is this just me?

                                                                    1. 1

                                                                      Well, switching applications with alt-tab is nothing like having multiple workspaces set up for quick switching. If all I had was alt-tab, I would also want multiple monitors (and indeed this was the case when I was stuck on macOS.)

                                                                      For one thing, alt-tab forces you to cycle through all applications, which is the biggest time kill.

                                                                      Second, alt-tab does not actually do anything until you’ve released the modifier key. That’s a long-ass time to spend on context switching. You would want your workspaces to be set up such that the switch happens as soon as you press the corresponding key, not when you release the modifier.

                                                                      Third, workspaces are different from individual applications. Workspaces are entire configurations of windows, much like multiple monitors give you.

                                                                      Fourth, in case it wasn’t obvious at this point: it needs to be really fucking fast. Faster then you can yaw your head. I haven’t tried to clock my workspace switching, but I know if I hold down the “previous workspace” key, it toggles back and forth between the last two workspaces and successfully renders both alternately. Autorepeat rate for my keyboard is set to 25, so switching definitely happens in less than 1000/25 = 40 ms.

                                                                      Alt-tab cycling on a single workspace is nothing like switching directly and quickly to a screenful window configuration.

                                                                      1. 2

                                                                        I’m not 100% sure, but based on some things you’ve said (“instead of being stuck to your sides” and “alt-tab forces you to cycle through all applications” in particular) suggest you aren’t using a tool that makes efficient use of multiple monitors. On Linux, this is understandable, because many of the Free Desktop standards (ICCCM and EWMH) are really inflexible when it comes to multiple monitors. And AFAIK, there’s been virtually no evolution in this space. The only way to make it better is to break the EWMH spec in a couple key places.

                                                                        In particular, window managers like the one found in GNOME are just absolutely abysmal IMO when it comes to multiple monitors. So much so, that a lot of their advantages are negated for my own workflow. The main problem is that you can only view one workspace at a time and each workspace covers all monitors. That is precisely the model that is enforced by EWMH. A much better model, IMO, is to have one workspace on each monitor with the ability to pull any workspace to any monitor you want. From there, is naturally follows that alt-tab should be among applications on a single workspace, which in this new model, will just be among applications on the currently focused monitor.

                                                                        This is exactly what motivated me to spend a couple years of my life building WIngo.

                                                                        When I’m forced to use GNOME, then I hack around some of its limitations with scripts. This one, for example, makes it easy to switch focus between monitors while respecting the relative stacking order of windows on each monitor.

                                                                        It’s really a pity what EWMH has done to multi-monitor support on Linux. It’s really held it back from being a lot better than it is today.

                                                                        1. 1

                                                                          Yes, you misread my comment. It was in reference to the comment above that, where someone explained how they find alt-tab insufficient on a single monitor, which I agree with. Multiple monitors never even entered the discussion!

                                                                          That said, I appreciate that you shared the information. I agree with most (if not all) of it, and this is also the reason I’m running XMonad, which handles workspaces in a non-EWMH compliant way too. (Maybe even very similarly to Wingo?)

                                                                          1. 2

                                                                            Ah sorry about that. I thought I might, hence the hedging. :)

                                                                            And indeed. XMonad’s handling of multiple monitors is what directly inspired Wingo’s model!

                                                                            I have fond memories of learning Haskell back in the day just to configure XMonad. Good times.

                                                                        2. 1

                                                                          My point doesn’t have anything to do with alt-tab in particular, though - it has to do with mapping different windows to different physical spaces. Workspaces, like you described, still have exactly the same problem - if anything, they make the problem worse, because now you have several different workspaces overlapping the same physical space, and then several different windows per virtual workspace. Additionally, my point is about total cognitive effect, not just the physical speed of whatever method you use to switch windows versus turning your head.

                                                                    2. 1

                                                                      I wanted to save some bucks on my electrecity bill so I switched to a single low-power monitor (nothing fancy, just a 22”) from a 3-screens setup + 1 laptop. And, honestly, this new setup is fine. I use Gnome 3 with the “Activities” button to switch between apps, and it works surprisingly well for me (I do programming, mainly).

                                                                      I have some theories, but I can’t explain why the single monitor setup works so well.

                                                                      1. 2

                                                                        I’ve found it to be a good approach for better focus. Also it makes it easier to transition from office to train - I’m not finding the lack of multimonitor a weakness.

                                                                        Not that train travel is relevant at this moment in time of course!

                                                                        1. 2

                                                                          How much electricity do you really save by switching from 3 monitors to one? I mean, electronics tend to be less power consuming than stuff like showers, heating, lights (if not LED), fridge, etc.

                                                                        2. 1

                                                                          Here in home office, I miss the second monitor at work. The big thing is remote meetings: With two monitors I can present one and prepare something on the other. With one monitor everybody sees me scrolling through my mails. It isn’t about privacy but about the distraction. Fortunately, confidentially is not that big a topic on my level.

                                                                        1. 2

                                                                          That’s a great post now, would be even greater 3 years ago when I was working a lot with python and multiprocessing =P

                                                                          I’ve bitten by pretty much all issues mentioned, specially signal handlers and timeouts. To make matters worse, it was a twisted app that also used lots of multiprocessing, so, double the pain. At the time we tried to move towards using less threads/processes and more async, but if I had this post back then I might’ve done it the other way around!

                                                                          1. 4

                                                                            Why still using twisted in 2020 when asyncio is part of the standard library or you can use more advanced async libraries like Trio or Curio?

                                                                            1. 3

                                                                              What’s wrong with twisted? It’s battle tested and works well.

                                                                              1. 1

                                                                                I have been used heavily years ago and I hate him :) The deferred/callback/errback mechanism is very powerful, but the code readability and code maintenance is terrible.

                                                                                I find asyncio easier to mantain and I really love Trio

                                                                                1. 1

                                                                                  For a while now you can use async/await with twisted. Makes the code a lot more readable.

                                                                                  1. 1

                                                                                    And the article shows that… :)

                                                                              2. 1

                                                                                Odds are, if it’s a networking or network-related problem, and someone else has encountered it in the past, Twisted will already have the solution either in Twisted itself or in its ecosystem.

                                                                                That’s pretty hard to beat.

                                                                              1. 1

                                                                                Here’s the thing I don’t get about “practicing for an interview”. There’s basically two scenarios in which an employer will want to hire me:

                                                                                1. To do things I know how to do well, and have done before
                                                                                2. To do things I’ve never really done before, but can probably learn how to do

                                                                                For each scenario, there’s two hiring process possibilities:

                                                                                a. They’ll ask me about things I’ll actually do in the job, or things I need to know to do the job well, or reasonable proxies to those things b. They will ask unrelated stuff

                                                                                For 1a, there’s no point in my practicing for an interview, because I’ll be setting myself up for failure. If they ask me about the things I’ll be doing if hired, and I don’t know them fluently enough to be able to answer without preparation, then I’ll not do well in the job. Better to fail now than 3 months in. A rejection email hurts less than being fired (source: been there, done that)

                                                                                For 2b, if they ask me about things I’ll be doing if hired, they’re morons, I clearly stated I don’t know how to do those things. They should instead ask about maybe somewhat similar things I’ve done, and try to find out if I can learn fast enough.

                                                                                For any b scenario, that’s an exclusion criteria from my part: If you want me to be a dancing monkey during the hiring process, it’s very likely I’ll not get enough autonomy and support to work well once hired, so I don’t really wanna work for you.

                                                                                Granted, this is not a perfect criteria: an employer might have what seems like a reasonable hiring process, in terms of not demanding ridiculous tasks, but also having a shitty culture. The hiring process is relaxed just because they’re hiring anyone that will take the job. But then again, that’s not my only criteria.

                                                                                Obviously, this comes from a quite privileged position of being able to take my time to find a job, every time I had to look for one. Maybe if you’re in a hurry, practicing is a better investment of time than waiting for a non-bullshit hiring process, since it will widen the pool of possible employers, even if at the risk of landing a not-so-great one (a developer’s gotta eat, after all).

                                                                                1. 59

                                                                                  I don’t want in the least to be a jerk about this piece. I can relate to it, inasmuch as my last effort to find a new job was for the most part a miserable and disheartening experience, and I don’t personally have anything like the will to engage in good faith with the kind of bullshit that’s described here.

                                                                                  With that said, this kind of thing - of which I’ve read countless iterations by now - mostly functions for me as a reminder that, at root, the industry is completely fucked and somehow we’re all playing along. The fundamental work of most “software engineering” is poisonous to the world, the organizations described in these essays are owned by people who fall somewhere on a spectrum between con artists and malevolent oligarchs, and a class of diligent strivers is engaged in competing for a place in the second-order technical/administrative ranks. A status that lets you earn much better money than in most of the rest of the economy while directly helping engineer everybody else into a state of precarity. (A state which, of course, is what awaits you if you exit the predatory scene you’re enabling before extracting enough resources to be insulated from its effects.)

                                                                                  Doing well in this environment seems to require either a quietly mercenary attitude or a profound capacity for self deception. There is of course a lot of human wreckage along the way. It’s striking to me how seldom I see anyone make the leap from “trying to get these jobs was a bad time full of getting raked over the algorithmic live-coding coals” to “we should burn this entire structure to the ground”.

                                                                                  1. 14

                                                                                    To be fair though, software engineering isn’t a homogeneous field, and I’m not sure ‘most’ is poisonous to the world. I have certainly never worked on anything I would consider poisonous, enabling of some kind of evil, or similar. What I do now is targeted at helping people out of bad situations and get themselves more stable.

                                                                                    If you have some analyses of software engineering across the full spectrum that can back up what you’re claiming, I’d be interested to see them, as what I hear in articles like this just sound like they’re from a bubble I haven’t been near.

                                                                                    1. 11

                                                                                      what I hear in articles like this just sound like they’re from a bubble I haven’t been near

                                                                                      Those are exactly my feelings. For the last 10 years in the field I haven’t been anywhere close to the environment described in pieces like this, and yet every time they pop up people (mostly on the orange site though) always find it relatable and comment with their similar experiences.

                                                                                      Perhaps it’s just my own bubble, but it seems so alien and bizarre that it seems like an entirely different industry whatsoever. My first job was in a ~300-person company, and that was the last time I had a technical interview, and the last time someone asked for my resume. Everything since then was a “oh, we already know you from somewhere else/seen your code on github, that’ll do, when can you start”.

                                                                                      All this talk about having a Top Dog Company in your CV, and I’m yet to see anyone even bothering to look at mine. All this bragging about massive salaries if you can get there, and yet my friends at Google get about the same money as I do with me bouncing between small companies and niche startups. I don’t think I’m that unique here either – and it makes me wonder why do people choose to put up with that crap? Multi-stage interviews, GDPR-insulting background checks, and for what? A status symbol, bragging rights? In an industry that is so starved for competent engineers that it’s willing to please you left right and center if you ask for it?

                                                                                    2. 5

                                                                                      Your comment has generalizations such as

                                                                                      at root, the industry is completely fucked and somehow we’re all playing along. The fundamental work of most “software engineering” is poisonous

                                                                                      and your solution (even though I hope metaphorical) is unrealistic and violent:

                                                                                      “we should burn this entire structure to the ground”.

                                                                                      I think that comments like that invoke raw emotions which harm constructive discourse. It is very unlikely that the “the industry is completely fucked”. I guess strong language feels good sometimes but may leed to people just expressing more of it and either cherishing their helpless victimhood or go for (hopefully harmless) violence.

                                                                                      1. 2

                                                                                        It is very unlikely that the “the industry is completely fucked”.

                                                                                        This is undeniably a generalization. All the same, it encapsulates pretty well how I feel about the broad spectrum of the software economy, from smalltime undertakings to startups to bigcos to megacorps. There are places where writing code isn’t directly in service of anything particularly egregious, and employment relationships that are relatively benign, but both are still situated in a dysfunctional ecosystem in technical, social, and economic senses. It’s almost impossible at this stage for even the best of intentions and practices to escape this reality. To a first approximation, there is very nearly no ethical participation in software which is compatible with making a living at it.

                                                                                        So: Yes, I think, completely fucked. Many intelligent people of good will disagree with me, but as these feelings have solidified, I’ve also found I’m hardly alone in them.

                                                                                        I think that comments like that invoke raw emotions which harm constructive discourse.

                                                                                        Perhaps. I don’t have any desire to turn a venue like this one into more of a shouting match. On the other hand, when I read material like the linked article, I can’t help feeling that a lot of people would be better off with a straightforwardly negative view of the system within which they’re competing for status and access.

                                                                                        1. 1

                                                                                          Thank you very much for the thoughtful reply!

                                                                                          Do you have examples of professions with a better ecosystem in terms that you like?

                                                                                          I guess “burning down” wouldn’t work or would actually just create chaos that is unlikely to be filled with something better. But I’d be interested in how something better would look like.

                                                                                          1. 1

                                                                                            Do you have examples of professions with a better ecosystem in terms that you like?

                                                                                            I’m really not sure. I think we could stand to learn from the healthier aspects of the trades, academia, and actual engineering disciplines. But then these are realms with their own deep problems.

                                                                                            But I’d be interested in how something better would look like.

                                                                                            I imagine there’s a possible path from where we’re at to somewhere better, however unlikely. I’d like to see people at existing companies unionize & otherwise organize. I’d like to see new enterprises organized from the start as worker cooperatives or similar. Beyond improving their own working conditions, I’d like to see technical professionals use their relatively high status and power to better the lot of working people in general and reign in the predatory impulses of capital. (If nothing else by, like, forcefully declining to pour labor into said predation.)

                                                                                            I’m not very optimistic about any of that, but sometimes I read essays like Considerations Regarding Distributed Software and dream a little.

                                                                                        2. 1

                                                                                          I honestly do not understand how trying to get everyone to speak like a cartoon version of a british buttler is an improvement of discourse. It’s not like this is a children’s tv show either, nor is lobsters funded by Disney.

                                                                                          I don’t think policing speech to try to make everybody write like a lawyer is gonna make the site “better”. Might reduce the ratio of curse words, but it will also make everything boring to read.

                                                                                          1. 2

                                                                                            Oh, I don’t care much about curse words. I think that the comment overgeneralizes in a way that is not helpful for discussion.

                                                                                            That is subjective and you are free to disagree. If you tell me why, I might even change my mind ;)

                                                                                      1. 32

                                                                                        If you were to start a new commercial project today, what language would you choose?

                                                                                        I’d choose Python. It’s not the best language, but I know it very well, I know its ecosystem very well, I know I can “easily” find other engineers to help me (and affordable) if I need. So, simply put, Python would be the best tool to make money, in my case.

                                                                                        1. 5

                                                                                          and affordable

                                                                                          Hey, who you’re calling cheap!?

                                                                                          Joke aside, do you mean affordable on the sense of there’s lot of people that know python so the scarcity doesn’t drive up salaries above average, or python developer salaries are below average?

                                                                                          1. 2

                                                                                            It’s an established language and platform, so there are no surprises with cost. Python is now commonplace, and while it will go into decline at some point, it will be relatively gradual. People won’t be surprised. There are plenty of companies that are stuck with platforms that have declined more quickly, but still need to be maintained long-term. Some JS frameworks like Angular are here in some markets.

                                                                                            In terms of average salaries, there is an expectation that established platforms will not be able to sustain salaries at the level they were when the platform was up and coming. The next cool thing is somewhat unknown, so developers can justify higher salaries because they are supposed to be working on the higher risk frontier.

                                                                                            When it comes to an individual you need to be doing more than Python to get a higher than average salary. Or, be paid to work remotely for a company that can sustain a higher salary (e.g. get a closer to SF salary, but live in the Midwest). If salary isn’t your primary goal, you now have more opportunities to practice Python because of greater market penetration. If salary is your goal, then the best plan is to be closer to the bleeding edge, and have a broader/deeper skillset.

                                                                                            1. 2

                                                                                              People won’t be surprised.

                                                                                              Does no one get in trouble with clients or management due to the later costs of Python? Having to put effort into keeping the software working when Python makes backward-incompatible changes that other languages are less likely to make. Spending time after-the-fact on mypy when the software gets large. Dealing with some aspect of the program being too slow.

                                                                                              1. 1

                                                                                                You are taking that out of context, but I understand your perspective. My point is that there is no surprises from the ecosystem going away, or declining rapidly, which then has a knock-on impact on the market.

                                                                                                Python applications can have many surprises in terms of costs later on as you point out. That said, I think a large part of that is to do with the inability of people to set proper expectations.

                                                                                                Anyone with experience, should realise that the convenience of Python comes with cost. They should highlight these trade-offs in some manner with other stakeholders. Whether it’s stretching Python, or someone creating an over-engineered monstrosity with some other framework for a company with no expertise in it, it really comes back to professionalism. There are real costs when we avoid discussion of the issues, or don’t document them, but people seem to press on regardless.

                                                                                                At other times we have to just accept that people with relatively little experience will make important decisions. We have to accept them and develop a plan for resolving the issues. If these decisions are happening continuously around us, then moving on, or developing better influencing skills seem to be the only options.

                                                                                        1. 3

                                                                                          The pattern that stands to me from both this post and my experiences with bad, ugly, and good standup meetings is that if you’re not directly producing deliverables, you should not be in the standup meeting. It’s goal is to coordinate between the team, not do status report. If the manager/product owner/whatever wants status updates, get it elsewhere: JIRA, git, email, chat, smoke signal, whatever.

                                                                                          1. 2

                                                                                            Standups should be for things that require communication between people. Some examples:

                                                                                            • blockers
                                                                                            • reaching out for help
                                                                                            • warning others about potential problems; surfacing info that would cost time or money if people didn’t know it
                                                                                            • giving advance notice about major interface changes (API, REST, method signatures, DB schema, whatever)

                                                                                            Telling 19 other engineers that you were editing module XYZ yesterday, when none of them have worked in that module in the last 3 months, nor will be in the next 3 months, is a waste of time. It does not involve more than one person.

                                                                                            1. 1

                                                                                              A 19 person team seems counterproductive already. I don’t think I’ve ever seen a team that big, let alone worked in one.