Threads for vosper

  1. 8

    “ To quote publicly available data, by 2020, we had around 2000 engineers with 20M lines of hand-written code (10x more including the generated code) in the monorepo alone, ”

    Every time I read stats like this I think - surely there must be a better way to write software!

    1. 7

      I agree! I hear a lot of good things about twitter’s culture (before Musk took over, that is). A kernel team, a culture of excellence, etc. But honestly, the actual service they offer is hosting a bunch of tweets, pictures, and videos. Their site fails to work very regularly.

      Surely there must be some challenges with scaling, but from the outside it just seems very crappy.

      1. 21

        I haven’t experienced a significant Twitter outage in years.

        I think you underestimate the challenges of running a near real-time many-to-many messaging service, to be honest.

        1. 12

          I think you underestimate the challenges of running a near real-time many-to-many messaging service, to be honest.

          And an analytics platform, a (very) high-volume data API platform for data customers, multiple search systems (some are paid-access only), an advertising platform, a content moderation system (across many countries, in compliance with local laws), probably a copyright infringement system, anti-abuse systems, internal monitoring and reporting, mobile applications, internationalization and localization systems, …

          People have this incredibly reductive view of what Twitter, in particular, actually does, that I find really frustrating. It’s not simple.

          1. 4

            People have this incredibly reductive view of what Twitter, in particular, actually does

            I have a vague memory of, possibly, a JWZ thing where he points out that browsers are huge now because, whilst you only use 10% of the functionality, B uses a different 10%, C uses a different 10% again, etc., and that leads to a complex system which is, by necessity, big.

            (But I cannot currently find the article / rant itself.)

          2. 4

            Nothing should require 20 million lines of code to accomplish.

            1. 8

              Why not?

              1. 2

                Because small things are exponentially easier to work with and understand than large things.

            2. 3

              Not an outage but we’ve all experienced breakage.

              1. 1

                I’m starting to have weird issues on my “Latest Tweets” timeline since a few days (on mobile).

                1. 1

                  I am excluding issues after the Musk takeover.

            3. 5

              So happy you said this. It seems like FAANGs get praised for their scale, when really it’s a completely pathological case of inefficiency that requires that many engineers to begin with. There is a better way, we can’t give up on that.

              What’s interesting is that works out to 10,000 LOC per engineer, which doesn’t sound like much but realistically how much code can a single human actually comprehend? LOC is not useful in many ways, but there is certainly an upper bound on how many LOC a human brain can reasonably work with.

              1. 5

                You can definitely write something that provides similar functionality in much fewer lines of code. I guarantee you won’t enjoy the on-call rotations, though.

                1. 1

                  This is confusing - are you saying that more lines of code implies better maintainability and reliability? That would go against any study where bugs are found to be very directly related to lines of code.

                  What I’m saying is that there’s an upper bound on physically how much code a human being can physically handle, and knowing what that limit is would be a good thing. I’m not suggesting that we play cos golf, but we should learn how to more efficiently use groups of people.

                  1. 1

                    It’s nothing to do with bugs per line of code. It’s how automated your procedures are for interfacing with the inherent complexity of hosting things in the real world. I’ve spent some years inside Azure dealing with this - the amount of effort it took to turn building & bringing online a new datacenter from a highly manual process to an even partially automated process was staggering.

                    1. 1

                      I see. Sure, if you want to solve problems, that comes with added logic. My criticism of large companies is that they can afford to have hoards (by which I mean ~tens of thousands) of humans add this logic at the abstraction level of our current tools, which hides the fundamental issue: I wish we’d be able to do equal or more with way less effort.

                      I understand that sounds pie-in-the-sky, but I’ve been at least experimenting with model-driven code generation a lot, and it feels slightly promising. Essential complexity can’t be avoided, but how much of the Azure datacenter code is essential?

                2. 2

                  We just had a post from someone who has a game with 58,000 LOC so 10,000 is likely to small: https://lobste.rs/s/lsspr7/porting_58000_lines_d_c_jai_part_0_why_how

                  1. 1

                    Sure, but it’s very interesting that it’s still in the same order of magnitude. It’s also very likely that individual productivity goes down on a multi-person team, because of the coordination necessary between people, and also because you have to work within parameters that you didn’t create.

                3. 4

                  I think the first step is to understand why it’s 20M lines in the first place. Is it lots of test code? Sad-path handling? Boilerplate? Features? Regulatory compliance? Maybe most of it actually is necessary!

                  1. 1

                    They had a kernel team and their own linux fork. I would bet on them having multiple MLoC of other forked / vendored deps too.

                  2. 4

                    It seems so, but I think it’s largely an illusion. Of course at 20M there’s probably a few M lines of code that could be cut, but I don’t think you could easily reduce it by an order of magnitude without losing something important.

                    Just like institutions grow bureaucratic processes out of past pains and failures, they grow code out of past pains and reaching breaking points with simple or out-of-the-box solutions.

                    For example, the original Twitter may have used off-the-shelf memcached, but grew to the point that its limitations around cache fragmentation and eviction strategies that don’t matter for most users, did matter to Twitter. Suddenly it’s a big step: from “just install memcached” it becomes “develop and maintain a custom cache cluster tuned to your workload”. Repeat that a few times, and you have millions lines of code that seem like they could be replaced with “simple” solutions, but actually everything would fall over if you tried.

                    Apart from scalability/performance, also resilience is a code multiplier. You not only develop feature X, but also error handling, lots of error handling, automated tests, healthchecks, fallbacks for every failure scenario, error handling for the fallbacks, rate limiting and circuit breakers for errors to avoid cascading failures, monitoring and logging, aggregation and alerting for the monitoring, and supporting infra for all of the extra code and tooling.

                  1. 5

                    Typescript is still sort of crazy though. You can still do absurd things (of course, since it’s really just js). What are some options if you just want a small compile to js language? And what are the experience using them?

                    Elm. Rescript. Purescript. Yew. They are all functional (not that I mind). I don’t know either how else I’d build a better dom. Sometimes I wish js would be stripped of 80% of its features so the lsp would stand a chance.

                    I’m just curious since I’ve sort of lost touch with the frontend world.

                    1. 2

                      Nim can compile to JS. I wouldn’t call it “small”, though. And I don’t know what counts as “crazy” to you.

                      1. 1

                        Yeah, I know it’s kind of a dumb question. I think my point was just that even ts isn’t great because it can’t restrict js. Crazy is being able to define methods on primitive types. Just as an example.

                        Didn’t know nim can compile to js. :-)

                        1. 1

                          If it helps, what you can do and what people actually do is pretty different. There certainly was a time (say around 2009) when you’d find popular libraries and individual developers extending or modifying the primitives, or messing with prototypes or globals, then clashing with other libraries that were trying to do the same thing…

                          That’s all gone now, and has been for a going on 10 years. Still possible, but no-one does it. In practice you don’t have to worry about it.

                          1. 1

                            I learned js around the time of prototype.js. On the one hand it makes sense (if you own and can control everything), on the other hand, not so much.

                            Css is similar, and it’s interesting to me that you now see small css frameworks pop up that straight up refines what, say, a p or h2 should look like. You could argue that if your website is small enough, and enough under your control that you can do that, then you could also go ahead and extend the prototype of primitive types.

                            All that said. I’m more interested in how a restricted language (that can work with the dom and events and all the difficulties of updating layout based on partially updating state) would look like. I wonder if anyone succeeded there. I think elm was (still is?) a good shot. As far as I can tell, all non-js attempts are functional.

                            1. 1

                              It depends what you mean by succeeds, but I think there’s a good argument that Elm did. If the language and ecosystem development hadn’t gone the way it did (closed to the world, then just abandoned [basically]) I think it really had a shot. There was a brief time when it wasn’t clear whether the future would be Elm, TS, Flow, or ReasonML.

                              Now it’s obvious TS is the winner but I do wonder if some kind of restricted subset of TS could be created that might meet your wishes.

                              1. 1

                                :-O didn’t realize elm had been abandoned. I see it. That’s sad.

                    1. 1
                      let [a, b, c] = input.split("|");
                      

                      Our array destructing gave us a false sense of security about the contents of our input string.

                      However, our destructing picks out the first three values of input.split(”|”), even if the resulting array contains more than three elements.

                      Don’t let lenient destructuring give you a false sense of security about the shape and contents of untrusted data.

                      It’s intentional and useful to be able to assign a portion of an array. If someone gets a false sense of security because they don’t correctly understand how their code works, it’s not the fault of the code. It’s not being “lenient”, because that implies there’s a stricter, better way it could be working. But this is destructuring assignment doing exactly what it’s supposed to.

                      1. 7

                        Is it designed to fracture, or not designed to unite? Both can cause the same conclusions but the framing is entirely different. I feel like it may be the latter.

                        1. 40

                          Defaulting to proprietary extensions only usable on the proprietary build of VSCode, and NOT the open source build? Definitely designed to fracture. The vast majority of users don’t know or care about the language integration extension licenses. They will just know that using OSS derivatives of VSCode means they can’t use any of the extensions they are used to.

                          I have a pretty favorable view of Microsoft these days, and I like to assume people generally act in good faith, but this is clearly embrace, extend, extinguish all over again. Or at least embrace, extend, become the de facto standard and gain the lions share of the market without technically extinguishing anything. As the author points out, high quality competitors like JetBrains exist and do well.

                          1. 2

                            What are they trying to extinguish? Jetbrains? Microsoft can’t really see Jetbrains as a threat. They wouldn’t even register on the C-level dashboard.

                            Microsoft has at least 400x Jetbrains’ revenue (based on 2020 numbers for both companies, it’s certainly a much bigger gap now), and this isn’t a market that’s going to grow by orders of magnitude like when they killed Netscape. I can’t see them bothering trying to extinguish anyone in the development tools space.

                            1. 17

                              Disclaimer: I work for MS, but I am not involved in VS Code (and don’t use it because the remote extension doesn’t work with FreeBSD or other unusual platforms and so I’d have to switch editors for a bunch of places where I can’t use VS Code and I don’t want to confuse my fingers).

                              My impression is that the strategy for VS Code and a bunch of other things from DevDiv is not about extinguishing anything, it’s about not being extinguished. Historically, Microsoft made great dev tools and these encouraged people to build things that locked folks into their platforms. Now, with Mac and Linux being common systems for developers to run, supporting Microsoft platforms (even Azure, which focuses quite heavily on Linux hosting) is often an afterthought. If someone develops a .NET service with VS Code, there’s a lot more of a chance that they’ll think of Azure Functions instead of AWS Lambda as a deployment platform (for example) than if they write it in a purely Linux or Mac-focused environment.

                              Again, this is just my impression, but Microsoft is not trying to extinguish F/OSS for a couple of reasons:

                              • These are the ecosystems where we hire engineers from. If we alienate them, our talent pipeline dries up.
                              • We make a ton of money selling cloud services to folks that want to run F/OSS. With confidential cloud computing, we can offer the kinds of very strong privacy guarantees to small players (even individuals) that you could previously get only with hosting your own infrastructure. F/OSS developers are a huge market of people who build services on the kind of infrastructure that we’re building.

                              I’ve had few conversations with CVPs who are still a bit confused about how the open source ecosystem works and how best to contribute, but they’ve all been positive about the idea. For a lot of things, our policy is to open source unless there is a compelling business reason not to (whereas, apparently, a few years before I joined to default was to keep proprietary and open sourcing needed business justification). That’s a huge attitude shift, but in a company of over a hundred thousand people it takes a while a propagate everywhere.

                              1. 2

                                Thanks for commenting on this. My (uninformed) take was similar - I think MS missed an entire generation of programmers for whom paying $$$ for a Visual Studio license was not only laughable - it wasn’t needed. They were making money not using MS tools.

                                I’m just a casual VS Code user, so I don’t know if there’s a market for proprietary, paid extensions. This would also track with MS historically - make it easy to code for MS products, and make money doing so.

                              2. 4

                                Microsoft is trying to extinguish free software in general, and this is just a part of it.

                                1. 2

                                  I don’t really believe that’s the case.

                                  1. 2

                                    I do. Just look at their actions.

                            2. 5

                              I would argue it is indeed designed to fracture because it’s open source(ish), encouraging others to adopt it for their own projects (like Gitpod, as detailed in the article). That said I’m not quite sure how I’d frame it as “not designed to unite” so perhaps I’m misinterpreting.

                              1. 1

                                Ha, that’s definitely a possible perception too :D. I like it.

                            1. 4

                              I’m responsible for approving cpu/ram/storage increase requests from developers and stories like these do kinda make me wonder if I should be as lenient as I am.

                              I pretty much approve every request because what else am I going to do? Scour the source code of every app for inefficiencies? I did do that once: someone who wanted 200GB of RAM just so they could load a huge CSV file instead of streaming it from disk.

                              Maybe it’s just a thing where trust can be built up or torn down over time.

                              1. 3

                                Asking from ignorance here: I’ve never worked somewhere where you had to request cpu/ram/storage. Instances or VMs, yes, but not asking to have some more RAM and having to say how much up front. How is that managed? You have processes killing containers that use more RAM than the developer asked for? Or more CPU? And… why? Is it a fixed-hardware environment (eg not in cloud) where it’s really hard to scale up or down?

                                1. 3

                                  Yes it’s fixed-hardware (grant funded), shared amongst several different teams. My role is mainly to prevent the tragedy of commons, and a little bureaucratic speed bump is the best I could think of.

                                2. 1

                                  Figure out how much the hardware will cost, figure out how much developer time will equal that cost, and force them to spend at least that much time profiling and optimizing their app before the request is approved?

                                  1. 1

                                    Problem in this case is that redis is (essentially) single threaded, so give it as many cpus as you might, if something is eating it at a high rate, you’ll need to solve the root cause.

                                    1. 1

                                      I know several people who worked at Yahoo in the 00s. To get new hardware you’d have to go to a hardware board, that included David Filo.

                                      He would grill you and then literally login to your servers to check utilization and if things didn’t look good enough he’d deny your request. In one case I was told he logged in and found miss-configured RAID controllers that wasted a bunch of resources. Request denied.

                                      I’m not suggesting you do this. But thought it was interesting.

                                      1. 5

                                        What an utterly bananas way for the cofounder to spend their time: hire people they don’t trust then micromanage their decision making.

                                      2. 1

                                        If you don’t look at the reason behind the request, then the process seems weird… If you don’t know what the app is doing, how can you decide if it should be approved?

                                        1. 2

                                          I mean, only investigating when something unusual is requested seems like a pretty reasonable heuristic.

                                      1. 1

                                        I know it’s unlikely the author will see this message, but because you never know who’s around: your quick links to :snap-target and snapChanging() and snapChanged() are broken.

                                        1. 3

                                          Sadly it seems a bit buggy to me - some of the UI doesn’t work in Chrome (save / apply) and it didn’t seem to parse any changes to my program until the timer had elapsed and I hit “restart”, which was confusing. Seems like it hasn’t been maintained since 2015.

                                          Pity, because I love the idea.

                                          1. 1

                                            I’m sort of hoping this post leads to someone sharing a fork or re-implementation that they know of.

                                            1. 3

                                              Yeah, me too. I’d especially love to see a Typescript version, because then there could be type-completion and help in the editor.

                                              I’ll take a look at the code, but I suspect this simulation is more complicated under the hood than it seems at first glance.

                                              1. 1

                                                Something with good error reporting for parse errors would be nice!

                                                The really surprising thing here for me is how well a naive solution works. Challenge 6 is the first one where a trivial algorithm doesn’t work.

                                              2. 1

                                                didn’t seem to parse any changes to my program

                                                Works pretty well for me when I press the Apply button

                                                1. 1

                                                  How do you tell that it’s done something? When I click it there’s no change that I can spot in the UI, and if my program has an error it doesn’t seem to show up until I restart (wish I wish I could do on-demand rather than waiting out the timer).

                                                  1. 1

                                                    When I press Apply, it does a restart for me. Based on what you’ve said, it sounds like it’s not doing that for you, but I couldn’t tell you why.

                                              1. 10

                                                There are a lot more methods of selection than just classes and IDs, and I find it incredible we don’t use more of them.

                                                Specifically, we should be using a lot less classes as we can infer a lot of our selection based on context. (i.e. main > article p {})

                                                1. 6

                                                  Could someone explain why, according to the BEM philosophy:

                                                  .button { /* stuff */ }
                                                  .button--state-success { /* stuff */ }
                                                  

                                                  is better than:

                                                  .button { /* stuff */ }
                                                  .button.state-success { /* stuff */ }
                                                  

                                                  Genuinely just want to understand the perspective.

                                                  1. 9

                                                    Because the latter increases specificity, making it harder to override, which leads to “specificity wars” as developers have to keep upping the specificity of their styles to override styles written by the last developer.

                                                    1. 2

                                                      Thanks, it looks like they have an FAQ for this question which gives some more detailed examples:

                                                      Why BEM modifer classes are prefixed

                                                  2. 4

                                                    Specifically, we should be using a lot less classes as we can infer a lot of our selection based on context. (i.e. main > article p {})

                                                    So, making evident that the CSS and HTML are very coupled. At that point you should realize BEM is too much work and it would be better to use TailwindCSS

                                                    Instead of <article class="card" data-align="left" data-size="small" />
                                                    you will write <article class="card text-left w-1/2" />

                                                    1. 8

                                                      So, making evident that the CSS and HTML are very coupled.

                                                      It’s not coupled. This selector need not match every page, or even any page. You write CSS that semantically marks up “the main article’s paragraph tags”, and if those exist, they get styled. If they don’t, they don’t.

                                                      One can have a general stylesheet applicable to many documents, and many different types of documents, and different structures of documents.

                                                      The stylesheet my blogs have used (in the past) were also usable with my articles extracted outside the blog template, or to printable form, or to emails. Not all styles are applied at all times. Only when they need to be, in the specific way they need to be. And in the pieces they need to be, cascading down.

                                                      Tailwind, on the other hand, is the exact opposite of CSS, and fails to separate presentation from data. You may as well just write inline style attributes instead of use Tailwind.

                                                      1. 3

                                                        Don’t put presentation in your markup when not necessary. Just <article> will do, then you style article elements.

                                                        1. 5

                                                          Why not? If we do what @boh suggested then the developer can see exactly how it’s styled right there where the article tag is created. Otherwise you have to go find the CSS that targets that tag (not always simple!) and then make the change there, hoping it doesn’t also affect something else. Unless you target by a unique classname or ID, which is still coupling your markup to your presentation, just with an annoying layer of indirection.

                                                          1. 7

                                                            see exactly how it’s styled right there where the article tag is created.

                                                            This is literally the exact opposite of what CSS was created for, and what we did in the HTML 1.0 days.

                                                            1. 6

                                                              I was around in the HTML 1.0 days. This is totally different. That was all table soup and image maps. Today we have a combination of semantic tags (article section nav etc ) and attributes (aria-whatever) plus semantically inert placeholders (div span). There’s no real reason to avoid the semantically inert placeholders. It’s not like you’re making a table and then using the table heading to hold the navbar anymore. It was bad because we had to use table for things that weren’t tables, not because everything needs semantics.

                                                              1. 5

                                                                Many things about the web have changed since CSS was created 25 years ago, I’m not that it’s intended purpose at that time is necessarily relevant.

                                                              2. 3

                                                                the developer can see exactly how it’s styled right there where the article tag is created.

                                                                Which is precisely what you don’t want. Styling independent of markup.

                                                                1. 1

                                                                  I’ve said a reason why I do want that, could you explain why you don’t?

                                                                  1. 3

                                                                    Making your markup independent of style makes it easier to change the style, both for you and for your users. And conversely, making stylesheets independent of markup makes them reusable.

                                                            2. 1

                                                              I am a Tailwind partisan, so I will criticize you from the other side, that “card” should just be shadow p-2 rounded border-2 border-gray-400 etc. :-)

                                                              1. 5

                                                                My understanding – please correct me if I’m wrong – is that this approach only works if you move the responsibility for defining “card” to a React component (or something similar), and so “re-use of the card concept” becomes “rendering the react card component” rather than “applying the card class”.

                                                                That is, from a first principles perspective, you still want the concept of “card” as reusable thing to exist… it’s just an argument about what method we use to implement?

                                                                1. 2

                                                                  Yeah, although it varies. Sometimes it’s good just to copy and paste because then you aren’t DRYing things together that don’t actually need to be DRYed together, since the visual similarity is just coincidental.

                                                          1. 1

                                                            This is just the yak-shaving parable retold in the form of home maintenance.

                                                            1. 3

                                                              It’s worse than normal yak-shaving, in that almost all of the time spent was completely unnecessary to solving the problem, which was unrelated to the task that required that a yak be shaved.

                                                              This story feels like debugging a legacy project you’ve never worked on before does to me.

                                                              1. 3

                                                                At a small startup with light read and write workload. Quite possible we don’t even have two users on the application at the same time, let alone writes happening concurrently at the DB level. We’re using Postgres 14 on RDS with whatever the defaults are. Most likely even if we got to a very high workload I wouldn’t change this. People on the team apart from me don’t know what transactions even are, let alone isolation levels etc…

                                                                Do you only use the default?

                                                                Yup.

                                                                Do you even care?

                                                                No

                                                                Do you change it when the transaction is only doing reads?

                                                                No

                                                                Finally do you do any type of application optimistic concurrency control (occ)

                                                                Nope.

                                                                1. 3

                                                                  But when you add full hints for a package used widely in your code base, you might find yourself facing an insurmountable mountain of errors.

                                                                  I don’t get this. If you add types in your code, and then add types from the third party libraries via stubs, wouldn’t that be a good thing and every error is an actual error?

                                                                  1. 4

                                                                    The reality is that a very large number of the errors that mypy finds are not bugs in your code, because mypy essential has its own (made up) type system which is only a subset of what python supports.

                                                                    Many of the “fixes” you are forced to do are actually different ways to “keep mypy happy”. This can be overwhelming to do all at once (and there is a bigger debate about whether it is ever worth it)

                                                                    1. 1

                                                                      I’ve been away from Python since just before typing came around, and I’m really surprised to read this. So are there two competing type-checkers, the official Python one and mypy? If that’s the case wouldn’t the official one (eventually) win out just by virtue of being official? There’s only one set of type annotation syntax, right? So to the extent that mypy doesn’t follow the official spec doesn’t that just make mypy wrong?

                                                                      Sorry for all the questions, I didn’t realise how much I’d fallen behind in goings-on in Python land.

                                                                      1. 1

                                                                        There is no official one! Python is dynamically typed. While type hints have been added to the language, there is very little built-in support for their meaning or functionality, no actual spec for what they mean, and certainly no compile-time or run-time checking in the core language or CPython implementation.

                                                                        There are several unofficial competing static type checkers, such as mypy and pyre.

                                                                        1. 1

                                                                          Mypy is part of the python GitHub account and I believe it’s maintained by the psf, so it’s the de facto official typechecker.

                                                                          1. 2

                                                                            I didn’t realise it’s now under the python account, that’s interesting.

                                                                            While it certainly seems to be the defacto standard, I’m not sure you could call it “official” in any way. For example, it’s barely mentioned in official Python docs, and it’s always at pains to call mypy a “third party” tool or one of several options. e.g.:

                                                                            This is important for things like how seriously you take mypy’s errors, and what is considered “correct” or “incorrect” usage of Python. If mypy tells me some code has an error in it, but it works at runtime, does that in any way imply my code is bad, or deprecated, or likely to fail in some future version of Python? As far as I can see, the answer is always an emphatic “no”, the static type system is a bolt-on which is not official in any sense.

                                                                            1. 1

                                                                              While it’s under the python org on GitHub, there’s no implication that mypy’s rules are somehow the official semantics of Python or can/should have an impact on those semantics.

                                                                              And things like the __class_getitem__ hook mentioned in the other reply were, at the time, controversial because originally the rule for type-hint-related features was that they weren’t supposed to require changes to the Python interpreter or to the language semantics, aside from the initial syntactic support for annotations.

                                                                        2. -1

                                                                          mypy essential has its own (made up) type system

                                                                          Unlike the natural, organic, type systems that grow on trees, in the forest?

                                                                          1. 3

                                                                            Unlike the Python’s type system, which utilizes protocol/trait style a lot, which is difficult to express in mypy type system. It’s possible, but as said, takes work.

                                                                        3. 1

                                                                          While I wouldn’t want describe it as a different system as other comments here, there are some really annoying things you can run into with mypy where mypy is slightly opinionated. One I completely disagree with mypy using specific type’s return rather than the parent one - for example there’s an IO type implementing the context manager contract which normally is expected to return any object (truthy/falsey value), but mypy says this class will always return None, because that’s cpython’s implementation and they don’t know of a usecase where it would differ.

                                                                          So mismatches like that can be found both for mypy and between users/providers of a library. Not too often and I still use mypy everywhere, but… there are annoyances.

                                                                          1. 4

                                                                            As an example of a “made up” type system, mypy accepts this:

                                                                            x = [1, "a"]
                                                                            

                                                                            but rejects this:

                                                                            x = [1]
                                                                            x.append("a")
                                                                            

                                                                            In Python lists are heterogenous, in mypy they are implicitly homogenous. You’ll also need to understand concepts like “covariant” and “contravariant” etc. - these concepts don’t exist in Python’s actual type system.

                                                                            Plus tons of other things, like “objects only have the attributes defined inside their __init__”, and “functions can’t have arbitrary attributes”, or “__add__ returns an object of the same type as the two inputs” etc,

                                                                            These are the kinds of things that you have to invent if you want to add a static type system to something like Python, but I think we do need to be clear that these are just mypy’s ideas about how code should be written and how a type system should work.

                                                                            1. 1

                                                                              That’s a list[int | str], you just need to annotate it.

                                                                              1. 2

                                                                                I think you misunderstood the criticism – the issue with the list above is that if mypy sees a single-element list it defaults to assuming the list is homogeneous and all elements must be of the same nominal type as the already-seen element. Which is not an assumption supported by Python’s semantics; mypy has basically decided to impose its own idea of how a list “should” be used in place of the real semantics of how Python’s list type works. The fact that you can annotate your way out of it doesn’t solve the problem with mypy – to see why, imagine if mypy instead assumed Python lists are fixed-length and forced you to annotate your way out of that to use append() on a list mypy thought was “full”. That would be a similar imposition of mypy’s preference over Python semantics.

                                                                                1. 1

                                                                                  Thank you for explaining that more clearly than I did!

                                                                                  I’m also not saying that mypy is necessarily “wrong” or bad in its decisions, just that we do need to be conscious of the fact that we are potentially facing a defacto change in what is considered correct or idiomatic Python.

                                                                        1. 3

                                                                          I opened this post expecting it to be about web components, but after reading it I’m not sure if it might be discussing some completely separate technology with the same name.

                                                                          However this time there’s probably little (if any) advantage to using web components for server-rendered apps. Isomorphism is complex enough; web components adds extra complexity with the need to render declarative shadow DOM and the still-unknown solution for bundling CSS.

                                                                          Server-side rendering is the idea that HTML should be generated on the server – the “old-school” approach used by everything from CGI.pm to PHP to Rails. Web components are a natural fit for this because they provide a way for the server to write out a description of the page as HTML elements, without having to generate all the nested <div>s and CSS rules and such.

                                                                          This person is talking about rendering shadow DOM on the server, though, which … honestly it seems completely insane. I don’t know why you’d ever do that, it’s such a bizarre idea that I feel I must be misunderstanding.

                                                                          The core flaw of component-oriented architecture is that it couples HTML generation and DOM bindings in a way that cannot be undone. What this means in practice is that: […]

                                                                          • Your backend must be JavaScript. This decision is made for you.

                                                                          Just absolutely befuddling. Why would using web components imply the presence of a backend at all, much less require that it be written in JavaScript?

                                                                          My blog’s UI is written with web components. Its “backend” is a bunch of static HTML being served by nginx. If I wanted to add dynamic functionality and wrote a web component for it, then the client side would only need to know which URLs to hit – the choice of backend language wouldn’t be relevant to the client.

                                                                          I’m building my version of this vision with Corset. I look forward to seeing what other solutions arise.

                                                                          Oh, ok, this is some sort of spamvertising. I looked at the Corset website and it looks like it’s a JavaScript library for using CSS to attach event handlers to HTML elements. The value proposition over … well, anything else … isn’t clear to me. Why would I want to write event handlers in a CSS-like custom language? Even the most basic JS framework is better than this.

                                                                          1. 5

                                                                            I re-read the article to see if the author was confused about “Web Components” vs. “components on the web”, and the answer is no. The author links to a WC library they wrote that mimics React, showing familiarity with both kinds of C/components. If you read closely, the terminology is consistent, and “web component” is used any time the author means using the customElement and Shadow DOM APIs, but “component” is used other times for the general idea of “something like React”. Frankly, it is extremely unfortunate that the customElement and Shadow DOM promoting crowd have hijacked the term “Web Component” for what they are doing, but the author is not a victim of this confusion.

                                                                            It’s a straightforward argument:

                                                                            However this time there’s probably little (if any) advantage to using web components for server-rendered apps. Isomorphism is complex enough; web components adds extra complexity with the need to render declarative shadow DOM and the still-unknown solution for bundling CSS.

                                                                            “Server-rendered” in this case means the server sends you a webpage with the final DOM. Your blog is not server rendered. If you disable JS, most of the content on your blog goes away. This is a standard terminology in JS-land, but it’s a bit odd, since all webpages need to come from some server somewhere where they are “rendered” in some sense, but “rendering” in JS-land means specifically inflating to the final DOM.

                                                                            Your backend must be JavaScript. This decision is made for you.

                                                                            That follows from the idea that you want to have <blog-layout> on the server turn into <div style="height: 100%"><div id="container">… and also have the <blog-layout> tag work in the browser. In theory, you could do something else with like WASM or recompiling templates or something, but so far no one has figured out an alternative that works as more than a proof of concept.

                                                                            Oh, ok, this is some sort of spamvertising

                                                                            Literally everything posted on Lobsters that does not have the “history” tag is spamvertising in this sense. It’s a free JS framework, and yes, the author is promoting it because they think it’s good.

                                                                            I find the idea of a CSS-like declarative language interesting, but looking at the code examples, I still prefer Alpine.js which also has a declarative language but sprinkled in as HTML attributes. I’m glad someone is looking at new ideas though, and I hope the “write a language like CSS” idea sparks something worth using.

                                                                            1. 2

                                                                              I’m still horribly confused even after your elucidations. Granted, I’ve never used Web Components, but I’ve read the full MDN docs recently.

                                                                              The bit about “your backend must be JavaScript” confuses me the most. Why? My server can generate HTML with any template engine in any language. The HTML includes my custom component tags like blog-layout, entry-header, etc. At the top of the HTML is a script tag pointing to the JS classes defining those tags. Where’s the problem?

                                                                              1. 3

                                                                                At the top of the HTML is a script tag pointing to the JS classes defining those tags. Where’s the problem?

                                                                                I think this is the problem the author points to, if you turn off JS, you don’t have those tags anymore.

                                                                                1. 1

                                                                                  No, the problem has to do with requiring JS on the server.

                                                                                  I’m, frankly, uninterested in what happens if someone turns off JS in their browser. I imagine that, unsurprisingly, a lot of stuff stops working; quel horreur!. Then they can go use Gemini or Gopher or a TTY-based BBS or read a book or something.

                                                                                  1. 2

                                                                                    It is both problems. The first view is blocked until JS loads, which means it is impossible to load the page in under one second. To remove the requirement that JS has loaded on the client you pre-render it on the server, but pre-rendering requires JS on the server (Node, Bun, or Deno).

                                                                                    I love you guys but this is a very old and well known topic in frontend. It’s okay to be backend specialists, but it’s not a confusing post at all. It’s just written for an audience that is part of an on going conversation.

                                                                                    1. 2

                                                                                      Fair, frankly I had hard time deciphering the blog post.

                                                                                      I agree though that optimizing for no JS is not interesting to me either.

                                                                                  2. 2

                                                                                    It’s important to know what the competition to WC is doing. Popular JS frameworks like Next for React and Nuxt for Vue and Svelte Kit for Svelte etc. let you write code that works both server side and client side. So if you write <NumberShower favorite="1" /> in Vue on server, the server sends <div class="mycomponent-123abc">My favorite number is <mark class="mycomponent-456def">1</mark>.</div> to the browser, so that the page will load even with JS disabled. Obviously, if JS is turned off, then interactions can’t work, but the real benefit is it dramatically speeds up time to first render and lets some of the JS load in the background while displaying the first round of HTML. (See Everyone has JavaScript, right?.)

                                                                                    To do this with WC, you might write <number-shower favorite="1"></number-shower>, but there’s no good way to turn it into first render HTML. Basically the only realistic option is to run a headless browser (!) and scrape the output and send that. Even if you do all that, you would still have problems with “rehydration,” where you want the number-shower to also work on the client side, say if you dynamically changed favorite to be 2 on click.

                                                                                    The Cloak solution is you just write <div class="number-shower">My favorite number is <mark class="number">1</mark>.</div> in your normal server side templating language, and then you describe it with a CSS-like language that says on click, change 1 to 2. Alpine.js and Stimulus work more or less the same way, but use HTML attributes to tell it to change 1 to 2 on click.

                                                                                    1. 1

                                                                                      To do this with WC, you might write <number-shower favorite="1"></number-shower>, but there’s no good way to turn it into first render HTML […] The Cloak solution is you just write <div class="number-shower">My favorite number is <mark class="number">1</mark>.</div> in your normal server side templating language

                                                                                      It’s possible I’m still misunderstanding, but I think you’ve got something weird going on in how you expect web components to be used here. They’re not like React where you define the entire input via attributes. The web component version would be:

                                                                                      <number-shower>My favorite number is <mark slot=number>1</mark>.</number-shower>
                                                                                      

                                                                                      And then when the user’s browser renders it, if they have no JS, then it renders the same as if the custom elements were all replaced by <div>. It’s not super pretty unless there’s additional stylesheets, but it’s readable as a plain HTML page. Go to my blog in Firefox/Chrome with JS disabled, or heck use a command-line browser like w3m.

                                                                                      1. 1

                                                                                        They’re not like React where you define the entire input via attributes.

                                                                                        No, that’s totally a thing in Web Components. It’s a little tricky though because the attributes behave different if you set them in HTML vs if you set them on a JS DOM Node. You use the lifecycle callbacks to have attributeChangedCallback called whenever someone does el.favorite = "2".

                                                                                      2. 1

                                                                                        I just saw Happy DOM, which can prerender Web Components server side without a headless browser. Cool! Still requires server side JS though, and there’s still a big question mark around rehydration.

                                                                                      3. 1

                                                                                        The keyword is “isomorphic”, which in “JS land” means the exact same code is used to render HTML on the server side and on the client side. The only language (ignoring WASM) they can both run is JavaScript.

                                                                                        1. 1

                                                                                          So their point is “if you want to use the same code on client and server it has to be JS”? Sounds like an oxymoron. But what does that have to do with Web Components?

                                                                                          1. 2

                                                                                            If you want to be isomorphic, you can’t use Web Components™, because they are HTML elements designed to work inside a browser. However, you can use web components (lowercase), e.g., React components, because they are just a bunch of JavaScript code that generates HTML and can run anywhere.

                                                                                      4. 2

                                                                                        Frankly, it is extremely unfortunate that the customElement and Shadow DOM promoting crowd have hijacked the term “Web Component” for what they are doing

                                                                                        Have they? I’m only familiar with the React ecosystem, but I can’t recall ever seeing React components referred to as “web components”.

                                                                                        1. 1

                                                                                          I’m saying “Web Components” should be called “Custom Element components” because “Web Components” is an intentionally confusing name.

                                                                                          1. 2

                                                                                            Oh I see, yeah, a less generic name than “Web Components” would have been a good idea.

                                                                                            1. 1

                                                                                              To be fair, Web Components as a name goes back to 2011, which is after Angular but before React.

                                                                                        2. 1

                                                                                          If you disable JS, most of the content on your blog goes away.

                                                                                          Did you try it, or are you just assuming that’s how it would work? Because that’s not true. Here’s a screenshot of a post with JS turned off: https://i.imgur.com/jLFz6UV.png

                                                                                          “Server-rendered” in this case means the server sends you a webpage with the final DOM.

                                                                                          What do you mean by “final DOM”?

                                                                                          In the cast of my blog, the server does send the final DOM. That static DOM contains elements that are defined by JS, not by the browser, but the DOM itself is whatever comes on the wire.

                                                                                          Alternatively, if by “final DOM” you mean the in-memory version, I don’t think that’s a useful definition. Browsers can and do define some HTML standard elements in terms of other elements, so what looks like a <button> might turn into its own miniature DOM tree. You’d have to exclude any page with interactive elements such as <video>, regardless of whether it used JavaScript at all.

                                                                                          That follows from the idea that you want to have <blog-layout> on the server turn into <div style="height: 100%"><div id="container">

                                                                                          If I wanted to do server-side templating then there’s a variety of mature libraries available. I’ve written websites with server-generated HTML in dozens of languages including C, C#, Java, Python, Ruby, Haskell, Go, JavaScript, and Rust – it’s an extremely well-paved path. Some of them are even DOM-oriented, like Genshi.

                                                                                          The point of web components is that it acts like CSS. When I write a CSS stylesheet, I don’t have to pre-compute it against the HTML on the server and then send markup with <div style="height: 100%"> – I just send the stylesheet and the client handles the style processing. Web components serves the same purpose, but expands beyond what CSS’s declarative syntax can express.

                                                                                          1. 2

                                                                                            Did you try it

                                                                                            Yes, actually. I tried it and got the same result as the screenshot, which is that all the sidebars and whatnot are gone and the page is not very readable. Whether you define that as “most of the content” is sort of a semantic point. Anyhow, it’s fine! There’s no reason you should care about it! But tools like Next and SvelteKit do care and do work hard to solve this problem.

                                                                                            What do you mean by “final DOM”?

                                                                                            Alternatively, if by “final DOM” you mean the in-memory version, I don’t think that’s a useful definition.

                                                                                            The DOM that the browser has after it runs all the JavaScript. “Server Side Rendering” is the buzzword to search for. It’s a well known thing in frontend circles. There are a million things to read, but maybe try https://www.smashingmagazine.com/2020/07/differences-static-generated-sites-server-side-rendered-apps/ first. The point about the browser having its own secret shadow DOMs for things like videos and iframes is true, but not really relevant. The point is, does the HTML that comes over the wire parse into the same DOM as the DOM that results after running JS. People have gone to a lot of effort to make them match.

                                                                                            If I wanted to do server-side templating then there’s a variety of mature libraries available.

                                                                                            Sure! But some people want to use the same templates on the server and the client (“isomorphism”) but they don’t want to use JavaScript on the server. That’s a hard problem to solve and things like Corset, Stimulus, and Alpine.js are working on it from one angle. Another angle is to just not use client side templating, and do the Phoenix LiveView thing. It’s a big space with tons of experiments going on.

                                                                                        3. 1

                                                                                          My blog’s UI is written with web components.

                                                                                          <body>
                                                                                              <blog-layout>
                                                                                                  <style>
                                                                                             ...
                                                                                              <yuru-card>
                                                                                                  <h2 slot=title>
                                                                                             ...
                                                                                          

                                                                                          Looks cool. Can you tell more?

                                                                                          1. 1

                                                                                            Anything specific you’re interested in knowing?

                                                                                            I use a little shim library named Lit, which provides a React-style wrapper around web component APIs. The programmer only has to define little chunks of functionality and then wire them up with HTML. If you’ve ever used an XML-style UI builder like Glade, the dev experience is very similar.

                                                                                            After porting my blog from server-templated HTML to web components I wanted to reuse some of them in other projects, so I threw together a component library (I think the modern term is “design system”?). It’s called Yuru UI because the pun was irresistible.

                                                                                            The <yuru-*> components are all pretty simple, so a more interesting example might be <blog-tableofcontents>. This element dynamically extracts section headers from the current page and renders a ToC:

                                                                                            import { LitElement, html, css } from "lit";
                                                                                            import type { TemplateResult } from "lit";
                                                                                            import { repeat } from "lit/directives/repeat.js";
                                                                                            
                                                                                            class BlogTableOfContents extends LitElement {
                                                                                            	private _sections: NodeListOf<HTMLElement> | null;
                                                                                            
                                                                                            	static properties = {
                                                                                            		_sections: { state: true },
                                                                                            	};
                                                                                            
                                                                                            	static styles = css`
                                                                                                    :host {
                                                                                                        display: inline-block;
                                                                                                        border: 1px solid black;
                                                                                                        margin: 0 1em 1em 0;
                                                                                                        padding: 1em 1em 1em 0;
                                                                                                    }
                                                                                                    a { text-decoration: none; }
                                                                                                    ul {
                                                                                                        margin: 0;
                                                                                                        padding: 0 0 0 1em;
                                                                                                        line-height: 150%;
                                                                                                        list-style-type: none;
                                                                                                    }
                                                                                                `;
                                                                                            
                                                                                            	constructor() {
                                                                                            		super();
                                                                                            		this._sections = null;
                                                                                            
                                                                                            		(new MutationObserver(() => {
                                                                                            			this._sections = document.querySelectorAll("blog-section");
                                                                                            		})).observe(document, {
                                                                                            			childList: true,
                                                                                            			subtree: true,
                                                                                            			characterData: true,
                                                                                            		});
                                                                                            	}
                                                                                            
                                                                                            	render() {
                                                                                            		const sections = this._sections;
                                                                                            		if (sections === null || sections.length === 0) {
                                                                                            			return "";
                                                                                            		}
                                                                                            		return html`${sectionList(sections)}`;
                                                                                            	}
                                                                                            }
                                                                                            
                                                                                            customElements.define("blog-tableofcontents", BlogTableOfContents);
                                                                                            
                                                                                            const keyID = (x: HTMLElement) => x.id;
                                                                                            
                                                                                            function sectionList(sections: NodeListOf<HTMLElement>) {
                                                                                            	let tree: any = {};
                                                                                            	let tops: HTMLElement[] = [];
                                                                                            	sections.forEach((section) => {
                                                                                            		tree[section.id] = {
                                                                                            			element: section,
                                                                                            			children: [],
                                                                                            		};
                                                                                            		const parent = (section.parentNode! as HTMLElement).closest("blog-section");
                                                                                            		if (parent) {
                                                                                            			tree[parent.id].children.push(section);
                                                                                            		} else {
                                                                                            			tops.push(section);
                                                                                            		}
                                                                                            	});
                                                                                            
                                                                                            	function sectionTemplate(section: HTMLElement): TemplateResult | null {
                                                                                            		const header = section.querySelector("h1,h2,h3,h4,h5,h6");
                                                                                            		if (header === null) {
                                                                                            			return null;
                                                                                            		}
                                                                                            
                                                                                            		const children = tree[section.id].children;
                                                                                            		let childList = null;
                                                                                            		if (children.length > 0) {
                                                                                            			childList = html`<ul>${repeat(children, keyID, sectionTemplate)}</ul>`;
                                                                                            		}
                                                                                            		return html`<li><a href="#${section.id}">${header.textContent}</a>${childList}</li>`;
                                                                                            	}
                                                                                            
                                                                                            	return html`<ul>${repeat(tops, keyID, sectionTemplate)}</ul>`
                                                                                            }
                                                                                            

                                                                                            I’m sure a professional web developer would do a better job, but I mostly do backend development, and of my UI experience maybe half is in native GUI applications (Gtk or Qt). Trying to get CSS to render something even close to acceptable can take me days.

                                                                                            That’s why I love web components so much, each element has its own little mini-DOM with scoped CSS. I can just sort of treat them like custom GUI widgets without worrying about whether adjusting a margin is going to make something else on the page turn purple.

                                                                                        1. 3

                                                                                          Nice, and a good overview of top contenders in the config lang space. There is one more worth mentioning, although it’s still in early development (but quite usable): Nickel from Tweag labs; the GH repo is here. Nickel is similar to Cue, but seems to go a bit further… it’s more of a complete programming language, optionally strongly typed, and adds ‘contracts’ as a related concept for verification. Looks very promising.

                                                                                          1. 2

                                                                                            If someone is looking to use a complete programming language for infrastructure they could do a lot worse than using Pulumi with Python, Go, Typescript, or Java. I’ve been using it with Typescript and it’s been great.

                                                                                          1. 3

                                                                                            This is pretty awesome and I found myself thinking that Java has gotten a lot nicer since I last used it. Then I saw this identifier: Executors::newVirtualThreadPerTaskExecutor. It took the whole width of my phone. Glad to know Java didn’t lose itself while pursuing new concurrency models!

                                                                                            1. 4

                                                                                              It’s a pretty good and descriptive name, though? It’s not exactly AbstractVirtualThreadFactoryBeanExecutorFactory

                                                                                              1. 2

                                                                                                Yeah, just a bit shocking that it took up the width of my screen.

                                                                                              2. 1

                                                                                                :-)

                                                                                                Over time, I am finding myself to prefer longer descriptive names. This works well for me when touching an unknown code base, or working simultaneously with multiple programming languages.

                                                                                                Avoiding language-specific syntax shortcuts, and code-base specific short-cuts in names – helps, in my view

                                                                                                As language and its standard library offers more and more patterns/constructs – I would expect this kind of long-nameness increase (and I hope do not go in the way of the pre-computer mathematical symbolism in math paper, that make them impenetrable for non-specialist reading).

                                                                                                May be there is a way to add font attributes, for example, or other visually-informative cross-language conventions (that work across all statically-typed languages).

                                                                                                I am not sure that using tabs as a ‘syntax-active’ construct is such a great foray into this area, but I guess, I appreciate, Python’s (Guido’s ) forethought in this area.

                                                                                              1. 6

                                                                                                Better title: a minor Windows program is a lot slower than it should be, for this user.

                                                                                                1. 35

                                                                                                  Given his track record of identifying performance problems in Windows apps, and not just minor ones. which mostly boil down to “the app is doing way to much, often unnecessary, work”, I think he is entitled to use this title.

                                                                                                  1. 1

                                                                                                    That may well be, but this specific article in isolation (which is what most of us are judging by) does not substantiate the claim. The author even acknowledges it in the first paragraph:

                                                                                                    I apologize for this title because there are many things that can make modern software slow. Blindly applying one explanation without a bit of investigation is the software equivalent of a cargo cult. That said, this post describes one example of why modern software can be painfully slow.

                                                                                                    1. 21

                                                                                                      I don’t understand the apology and think the title is fine. I read it like: “Why criminals are caught (part 38) – (the case of) the Hound of Baskervilles”. Where the parenthetical parts are optional. You wouldn’t think a blog post with that title would claim all criminals are caught because of the specific reason in that case.

                                                                                                  2. 11

                                                                                                    for this user.

                                                                                                    Are there users for whom waiting an extra 20 seconds before they can start using a program is acceptable?

                                                                                                    1. 7

                                                                                                      No, but there are likely users with fewer than 40,000 files in their Documents directory.

                                                                                                  1. 13

                                                                                                    About the only thing I disagreed with from skimming the article was that state machines are underrated. Automata theory is one of the core parts of a computer science degree and is used everywhere. Implementing FSMs is so common in software development that there are design patterns for it and even embedded DSLs in a load of languages. Protocol descriptions are almost always written as FSMs (an abbreviation that I always read as Flying Spaghettini Monster). Theorem provers and software verification use them extensively.

                                                                                                    So I read the initial claim in the same way that I’d read ‘conditional execution is underrated’. You don’t need it to justify the rest of the points.

                                                                                                    1. 25

                                                                                                      They might be widely used in theory, but they aren’t used enough in practice. I can think of a bunch of problems I encountered in my career that could have been solved much easier with state machines than the solutions I had to come up with simply because I did not know that state machines could be used that way.

                                                                                                      Out of all the posts that came out during the “wc but faster” fad, the one that stood out to me the most is how you can beat most of the implementations with predictable performance using a state machine. There is a lot more problems that can, and should be solved with state machines, but currently aren’t because they mostly stay in theory.

                                                                                                      1. 6

                                                                                                        I’d been following the “beating wc” thing pretty closely but haven’t seen the state machine solution before. Thanks for sharing!

                                                                                                        1. 4

                                                                                                          Yeah, I think in practice FSMs are under-utilized. They are all over the place in theory, but in industry, not so much.

                                                                                                          1. 2

                                                                                                            I’m really surprised to hear that. I can’t think of a single non-trivial project that I’ve worked on that hasn’t had a bunch of explicit state machines in it.

                                                                                                            1. 4

                                                                                                              This is my experience as well.
                                                                                                              Most nontrivial user workflows I have encountered tend to have either an explicit state machine, or a close-enough re-implementation of a state machine, built by an autodidact who found their way towards the basic ideas without knowing them by name.

                                                                                                              However, I can see how that might not be the norm.
                                                                                                              The difference between the experiences of you and I, and @ignaloidas, might well have been pure chance: Did the person(s) who stood up the project initially recognize the state machine at the center of it all, or did they not have the awareness of them to do so? I have also been the one to introduce several junior and mid-level engineers to the concept. It is possible that some of their past projects would have benefited from this tool that they didn’t have.

                                                                                                              1. 9

                                                                                                                My experience is the exact opposite: in startupcanistan, you usually are running with cheap and comparatively inexperienced developers that don’t have the CS background to articulate these sorts of problems.

                                                                                                                (Exception: game development, at least back in the 00s, had FSMs as like a super-common idiom for NPC AI. Webshits lack that background.)

                                                                                                                What I’ve seen happen is that you end up with a bunch of semi-articulated FSMs in the codebase (“let’s add a column to track if an order is being shipped, created, fulfilled, etc.”). Somebody eventually comes along and says “okay folks this is an FSM maybe we should make sure that’s driven by events and transitions”, and then either you get some astronaut who starts babbling about event sourcing and everything goes off the rails or you end up not doing it because of other spaghetti code that screws with the state directly (and which is too expensive to fix at the moment).

                                                                                                                I really cannot stress how big a gap there is between the vulgar software development that actually pays rent and the “basic” academic progress folks are more familiar with. It’s incredibly frustrating to straddle the two worlds and interact with people (some on this very site!) that seem to have little direct experience of the great unwashed masses and seemingly little willingness to make allowances for their existence.

                                                                                                                1. 13

                                                                                                                  Webshits

                                                                                                                  What’s with this term? Is it yet more of the tiresome sneering at people working on web systems that we see so much of in sites like Lobters or HN? Can we not just be nice?

                                                                                                                  1. 8

                                                                                                                    I see two aspects of this. One is dismissing the developers. The other is dismissing the environment. Dismissing people is obviously not nice, but the environment has a number of glaring issues that is holding everyone back.

                                                                                                                    The most significant aspect, I think, is that front end web stuff systematically sits on a huge pile of tech: HTML, CSS, and JavaScript are so far removed from the CPU that performance problems are very different, and I’m pretty sure much less predictable, than in native development. Hence the first thing: people working in front end dev won’t have the opportunity to learn what performance really is.

                                                                                                                    Backend is arguably different, but not that much. PHP, Ruby (on Rails), Node… all those are too far removed from the real capabilities of the hardware to really be perceived. But at least you may work with Go, Rust, or C++. There are more opportunities to learn about what the hardware is capable of.

                                                                                                                    Personally I hate the web, and want nothing to do with it beyond simple static web sites. It’s easy to fall into the trap of extending that hate to the developpers themselves, but in all likelyhood they just do what they can like everyone else.

                                                                                                                    1. 5

                                                                                                                      The other is dismissing the environment

                                                                                                                      Culture is why I quit Ruby on Rails after less than two years.

                                                                                                                      Ruby divorced from Rails is a great language. Rails, on the other hand often felt like playing in DHH’s conceptual sandbox, and I was surrounded by people who were telling me that my honed instincts for modularizing, reducing coupling, and taming complexity was wrong because it “wasn’t the Rails way” and “the web is different.” Of course, the web is really just a platform, and it isn’t magically different from any other platforms.

                                                                                                                      Those thought-terminating cliches were tribal markers that were elevated into legitimate engineering advice. And several Rails-based startups improvised solutions to conceptually scaling their Rails monolith, some even at the time.

                                                                                                                      The culture at large never had the ability to say, “we might have been wrong,” or even, “there is no one size fits all approach to architecture.” Such intellectual discontinuity is not healthy to continued growth as an engineer.

                                                                                                                      1. 3

                                                                                                                        “wasn’t the Rails way”

                                                                                                                        I had the same experience; trying to shoe-horn modularization en encapsulation onto Rails is a non-starter because it doesn’t fit the framework. When I was programming in Rails, everything I tried to do just hurt, and I later realised that’s because I was fighting the design of Rails every step of the way.

                                                                                                                        So there’s some truth to the pushback that it “isn’t the Rails way” because not doing things the Rails way and still trying to use Rails is setting yourself up for a world of hurt. Unfortunately.

                                                                                                                    2. 6

                                                                                                                      I have done software engineering on web sites and applications for more than a decade (or management thereof). I sometimes refer to myself as “nothing but a simple webshit” or “just a webshit, doing my shit on the web”, as a self-deprecating term.
                                                                                                                      When doing so, I use it to acknowledge the chaos of the environment (as @Loup-Vaillant describes), and to not act too proud of the variety of work. It has its complexities, for sure, but it is often not as professionally rigorous as some other varieties of software engineering.

                                                                                                                      I am sure that @friendlysock can speak for themselves, but being that they work (or have worked?) in that environment, I suspect that they are doing similar.

                                                                                                                      That said, I understand how it is a pejorative in the wrong sentence or in the wrong context.

                                                                                                                      1. 1

                                                                                                                        What’s with this term?

                                                                                                                        I believe I picked it up from n-gate, the most important thought leader/memento mori in our industry.

                                                                                                                        Is it yet more of the tiresome sneering at people working on web systems that we see so much of in sites like Lobters or HN?

                                                                                                                        Speaking as one of those people who has been working on web systems for many years (and who has done non web system work as well), I feel perfectly comfortable and justified in using that term. I myself have been, am, and–for the right price–will continue to be a webshit.

                                                                                                                      2. 8

                                                                                                                        A few years ago, after nearly 20 years as a professional software engineer, I got my first job for a webshit startup. I was amazed at how different it was from all my previous jobs. I think I was the only person in the whole company with a CS degree: all the other developers had done a few months in a “coding bootcamp” at most. But I was also one of the least knowledgeable employees when it came to product development: I can write assembly, compilers and network stacks but I had never heard of an Opportunity Solution Tree before.

                                                                                                                        It’s like there are two (at least) very different software industries. You can be an experienced expert in one of them without ever encountering the other one at all.

                                                                                                                        1. 1

                                                                                                                          I found that moving to a browser where my code would immediately just render neat things on screen was just such a nice change from all of the stuff I had to do in graphics and UI programming to get a tenth of the results. I found Javascript quite liberating compared to C and Java.

                                                                                                                          At the same time, I knew in the background how inefficient all of it was and the more I was exposed to “best practices” the fewer clothes I found the emperor to be wearing. Every family is unhappy in its own ways. :(

                                                                                                                          1. 2

                                                                                                                            I think the modern web (for some value of “modern”) is probably a very nice environment for GUI programming. Something like Chrome’s “Developer tools” turns the web browser into a pretty amazing, graphical, REPL-like, interactive dev tool. I suck at GUI development, sadly, but I was impressed by what I saw when I worked with some really talented front-end developers.

                                                                                                                            In general, the webshit startup world seems to be less about technology itself and more about using webshit tech to quickly add business value. When you can just push to master and a few minutes later your code is running in production, it really changes how you work. Product testing, for example, is qualitatively different when your product is a website compared to hardware: if it takes months to build and release a prototype, a product development iteration looks very different. It’s also very different when a bug just means that a website breaks and it can be fixed by redeploying in minutes, rather than having serious safety concerns and requiring a physical hardware recall.

                                                                                                                            This has knock-on effects on the whole business but I particularly enjoyed being part of a small product team for my day-to-day work. Every day, I would work closely with teammates who might be other software developers, or UX designers, or product managers. In the past, my team would be software engineers, hardware engineers, and maybe mechanical engineers. It was a nice change.

                                                                                                                            However, I found that I only really had autonomy to the extent that I had some control over the product development. I worked for one startup that did this really well, where it really did feel like we were all one product team: we might have different specialisms - some were designers, some were programmers - but we were all one team and we all had both the responsibility and the ability to control how the product was developed. But I also worked in a couple of places where someone more important than the code monkeys had worked out what needed doing and just drip fed us tasks to do. That sucked.

                                                                                                                            Without the product/business side of things to challenge me and offer me a sense of autonomy, I found webshit technology pretty dull. It’s all just ETL or CRUD APIs, both of which are pretty tedious. In contrast, in the non-webshit-startup parts of the software industry, I could find my challenges and a sense of autonomy in the technology itself: I’d be given a hard technical problem to solve and could go away and research it, design a solution, and implement it, usually over a few months at least.

                                                                                                                            I wouldn’t say either option is better or worse. They’re just different. I’m glad I’ve had the chance to do both.

                                                                                                                  2. 4

                                                                                                                    Agreed. I had a colleague once who got so excited by the concept of state machines, he built an elaborate one himself from scratch for a web app he was assigned to build, not realizing the framework he was using already had two state machines in the form of a router and state container (which is admittedly not a state machine on its own but can be implemented as one). It was only after he finished the project that he realized his state machine, like an inflamed appendix, was destructively superfluous.

                                                                                                                    1. 3

                                                                                                                      I love “destructively superfluous” - it’s the most pithily self-contained description I’ve heard of the crawling chaos it so neatly encapsulates. Keeping that, thanks ;-)

                                                                                                                    2. 2

                                                                                                                      I’m curious if you have a sense of what the lynch pin of seeing problems this way is? I have a cursory understanding of what state machines are and an incrementally-growing list of things they are or can be used for, but I’ve yet to have the realization that a state machine is a good fit while I’m in the trenches working on something.

                                                                                                                      My gut says that I need to intentionally apply the concepts a few times, but maybe that would be a waste of time if this is something where focused study and synthesis are more helpful?

                                                                                                                      1. 2

                                                                                                                        The first time I learned about state machines was in an undergrad algorithms course, and our prof emphasized how good a fit they were for a lot of text processing problems. He described them as a super-tool, secret weapon, or somesuch verbiage. That was spot on.

                                                                                                                        FSMs are applicable to all kinds of input processing, not just text. For example, I was once tasked to write a monitor for virtual machines. The whole thing was basically a big state machine and some functions for reading data from various OS sources.

                                                                                                                        Here’s another example from my undergrad days. I was taking a computer architecture course, and our first assignment was to trace the execution of a mock simplified CPU, given a set of assembly instructions. You were supposed to do it by hand. But I realized I could make a computer do my homework for me. The fetch-execute cycle is basically a state machine. I wrote a little emulator and wrote some lex to turn the text assembly code to machine code for my emulator. I didn’t even need yacc for that. I probably spent more time than I would have on the actual assignment, but getting a computer to do tedious stuff for me was more exciting.

                                                                                                                        Spend some time playing around with FSMs for simple text processing, and you will probably start to get a good idea of the shape of problems they can solve.

                                                                                                                        1. 2

                                                                                                                          In most places, the problem is expressed as a state machine (parsing, protocol models) and so implementing anything else requires more thought. Just translating the problem into code naturally produces a FSM. I guess part of my bias comes from the kinds of large codebases I’ve worked on:

                                                                                                                          • Compilers have a load of state machines, starting at tokenisation. Language runtimes are full of them for things like object lifetimes, GC phases, and so on.
                                                                                                                          • OS kernels are made almost entirely of state machines. Network protocols, socket, file descriptors, file and IPC mechanisms, and virtual memory are all basically state machines. A lot of these have to conform to a spec that is written as a state machine.
                                                                                                                          • GUI frameworks are full of state machines for things like drag and drop protocols.
                                                                                                                          • Hardware projects are basically all either state machines or pipelines. Hardware description languages typically have very nice tooling for producing state machines because they’re a huge part of the logic.

                                                                                                                          Many of the hints that are not simple FSMs in these projects are push-down automata (stacks of state machines).

                                                                                                                          One of the reasons that I liked Objective-C (and other Smalltalk-derived languages) is that they provide a really nice pattern for implementing state machines. You define a class with one subclass per state and one selector per message and the swizzle the isa pointer on each state transition, so each subclass just has the methods implemented for the valid transitions from that state and you get an exception if you try to perform an invalid transition (or your superclass implements default behaviours that transition to an invalid state and do some error recovery).

                                                                                                                          You can do this in languages like C++ with a delegate object and Boost also has some nice libraries that let you implement states with one class per state and static checks that impossible transitions can’t occur.

                                                                                                                          C programmers typically implement FSMs with an explicit state enum and a load of switch statements but pretty much every other language I’ve used (systems languages, application languages and hardware languages) either have nice design patterns or libraries with nice DSLs for implementing them.

                                                                                                                      1. 6

                                                                                                                        Richard joined NoRedInk in the fall of 2013 as an engineer. He introduced Elm to the stack in 2015, and later authored the book Elm in Action for Manning Publications. Today he works at NoRedInk on building the open-source programming language Roc, which can be found at roc-lang.org.

                                                                                                                        TIL NoRedInk is creating another programming language. Hats off to them for being willing to make the investment.

                                                                                                                        From the Roc FAQ

                                                                                                                        Roc is a direct descendant of Elm. The languages are similar, but not the same.

                                                                                                                        The Roc editor is one of the key areas where we want to innovate. Constraining ourselves to a plugin for existing editors would severely limit our possibilities for innovation.

                                                                                                                        And creating a new editor, too!

                                                                                                                        1. -3

                                                                                                                          It strikes me as absolutely crazy to name a project that’s unaffiliated with the Apache Software Foundation “apalache”.

                                                                                                                          It seems weirdly adjacent to typosquatting.

                                                                                                                          1. 1

                                                                                                                            I don’t think bikeshedding naming is really that useful.

                                                                                                                            I will add that I think plenty of people have never heard of ASF (I haven’t interacted with anything out of that org for 10+ years) and Apalache sounds and looks reasonably distinct from Apache to me. It also seems to be the Spanish name for the Appalachian mountains (and they have a mountain logo).

                                                                                                                            To sum up: meh, it’s fine. I really don’t think it’s “adjacent to typosquatting” - that suggests a kind of bad intent from the developers that I don’t think is justified.

                                                                                                                            1. 1

                                                                                                                              They used an ASF license for their project. They’ve clearly heard of ASF.

                                                                                                                              Their logo looks like the Apache Hudi project logo to me.

                                                                                                                              I didn’t intend to bikeshed the name so much as to point out that I was confused and surprised to see that it was unaffiliated.