1. 17
  1. 1

    I re-read some parts and I still cannot figure out who this person was or what was the goal.

    It was written as some sort of malicious actor who acts independently but how is that possible? How can someone even have the guts to do this unless they are management or some C-level person?

    I just cannot imagine someone going past all the people working on their part and wanting to deploy straight into production some random patch for no reason whatsoever.

    1. 10

      I’ve never been in such a situation myself, but it seems like the sort of thing that can happen when a company has multiple related services for different kinds of customer.

      Imagine ToDo Inc. provides to-do-lists-as-a-service. Most of their engineering effort goes toward scaling and maintaining their service for their millions of consumer users, but they also have a division that provides heavily customised to-do lists for enterprise users. The consumer and enterprise front-ends run on the same servers (because the software is very similar, and uses the same libraries to talk to mostly the same back-end servers) but the enterprise version of the system is deployed on a slower cadence and does more runtime lookups to determine what theme to use and what features to display.

      Most of the engineering staff will work on the consumer code, because that’s where the engineering need is greatest, but they’ll actively avoid learning anything about the enterprise code because it’s frustratingly political (“we normally do X because it’s faster, but customer A’s CEO specifically asked for a different thing so for that customer we do Y instead…”). Meanwhile, the enterprise-side engineers avoid learning about how scaling works because they don’t need it, and they’ve got their hands full dealing with the complexity of enterprise integrations.

      And then along comes an enterprise customer who wants to integrate data from their mainframe system into the front-end, and the managers allocate three months for the project because the “hard part” (i.e. negotiations with the other company) will take nearly that long, and the enterprise engineers assure them that there’s a magic “deploy to the entire fleet” button they can press at 3PM on the Sunday before go-live, so the technical part is easy.

      1. 1

        I guess this scenario makes more sense now.

      2. 6

        How can someone even have the guts to do this unless they are management or some C-level person?

        I haven’t seen anything quite this obnoxious myself, but I’ve definitely seen requests to deploy some fancy new feature or unexpected dependency at the last minute. Oftentimes this comes from a team that has to use the machines in question, but isn’t responsible for their maintenance and isn’t aware of any consequences.

        Really, all it takes is for someone to say, “what can it hurt to ask?” and try to get their change in. If you’re not responsible for owning the underlying system, it’s often unclear what changes are reasonable and what changes are dangerous.

        As an example: I’ve often managed multi-tenant scientific computing clusters with hundreds of users. More than once I’ve refused to let a single user install a root-user daemon on the full cluster, that would have helped their particular project but messed with everyone else. It’s generally possible to find an alternative that makes everyone happy, but it sometimes takes a while to teach people that production infra is not the same as their desktop…