Threads for achals

  1. 13

    We use vegeta at work and it’s pretty great.

    1. 3

      Importantly, it does not suffer from coordinated omission, which almost every other load generator does. Especially the ones you hack together on your own. Properly generating load is hard.

      Coordinated omission is when the load generator backs off on the pressure when the target is overloaded. It happens naturally since you can’t keep an infinite amount of outgoing requests active at once. You have to compensate for this effect.

      Coordinated omission sounds like something that will just result in overestimated performance numbers, but it’s worse than that. It can easily reverse figures, making performance improvements look like things got worse.

      1. 1

        It seems like the Coordinated Omission problem is more or less a euphemism for backpressure. Backpressure is a cross-cutting concern, though. You can see it if any of your systems in the chain slow down, from the load generator, through the web server, appserver, etc. A bottleneck in any area makes the downline systems look “bettter” under reduced load.

        The remedy for backpressure awareness is that, as you add more concurrency you watch that the graph for throughput and requests/second increase in the exact same manner as the increase in VUs. I don’t think this is a tool problem–just an area to have awareness. For example, as I ramp up load, if I’m using a step pattern, bandwidth and hits/second should follow the step pattern. If they don’t, the test is invalid the moment they deviate.

        The worst tools, in my experience, are the ones that “throw load” when real users would be “stuck.” Those tools are far more dangerous and have worse failure modes. For example, if the page returned happens to have a 200 status code, with a giant message saying “Critical System Failure” a tool that ignores backpressure and slings load might show you improved performance once the error page appears as it loads fast!

        1. 2

          Good points! With your awareness of the problems involved, and willingness to declare a test invalid when the actual load doesn’t match the attempted load, I’m not too worried about you drawing the wrong conclusions – no matter what tool you use.

          The problem you mention in your last paragraph is a problem whether or not your tool corrects for constant attempted load. Any tool ignores backpressure when the “backpressure mechanism” is a very fast 200 OK.

    1. 13

      Finish reading 1984. I’m halfway through and man it feels like it was written by an oracle.

      1. 3

        His nonfiction books The Road to Wigan Pier and Down and Out in Paris and London are also excellent.

        1. 1

          Homage to Catalonia is also essential. Perhaps the best work of long-form journalism.

        2. 1

          I have recently started reading his book Why I Write. It pulled the curtain back a little on books like 1984 and lead me to think the problems he writes about aren’t new, they just seem to have a fresh coat of paint in the modern world.

          Also one piece of advice given with little context to avoid any spoilers. Don’t skip reading the appendix. For anyone not worried about spoilers or without a copy on hand: Spoiler

          1. 2

            Which is of course funny because we use one orwellian term in our daily lives these days: social distancing.

            Misusing language seems popular.

          2. 1

            If you want another prophetic story J.G. Ballard’s The Intensive Care Unit is about social distancing.

            It’s available in The Complete Stories of J.G. Ballard.

            The article Why we are living in J.G. Ballard’s world discusses this story and some of his other works.

            1. 1

              Yeah, it’s a good one. I’d suggest reading some of Orwell’s other stuff too, or if you like a nice pairing go read Brave New World as a chaser…two vastly different approaches to dystopia.

              1. 1

                Thanks got the recommendation - will check it out

                1. 1

                  Two vastly different approaches to describe our modem day lives.

                  1. 1

                    I recently read Homage to Catalonia and was very impressed - Orwell talks about his experience as a soldier in the Spanish civil war and you can see how that experience so directly influenced his thoughts and writing.