1. 57

  2. 16

    seriously, how do you prevent stuff like this from happening if you have a big team. for small companies its relatively easy to prevent. but lets say you have 1000 engineers, how do you prevent 1 of them from making a mistake…

    1. 38

      One way to catch this sort of thing is sentinel data — in this case, you could use a unique value as a test account’s password and use that account for testing every service, then search everywhere you can think of for that value. If it shows up anywhere, the siren goes off. In enterprise storage similar things are done for “data loss prevention” to make sure people don’t move sensitive files to someplace they shouldn’t.

      1. 4

        I have never heard of those techniques but it seems interesting! Do you have some recommendation of good readings about this subject?

      2. [Comment removed by author]

        1. 2

          Even then. Divert full request logging on your SSL-terminating http proxy server and you’ve just caught a bunch of passwords in-flight. And you probably don’t want to use client-side crypto, or else you lose >1% of potential users. More in places with poor connectivity due to longer page loads.

          These things just happen and we need to live with them. When you let someone else handle your data, you should be able to trust them. If they do it on scale, there must be some regulation. Just like you trust your friend not to poison you with a dinner, but you prefer someone to check your favorite restaurant to maintain some standards.

        2. 4

          It can easily happen in a team of 10 people. I’m of the strong belief that someone on the team needs to be responsible as a security architect and that’s their main role.

        3. 8

          Sounds like they didn’t redact passwords from logging, ouch. They just published a brief official statement on this.

          1. 7

            You move fast and you eventually break things. That’s how it goes.

            1. 3

              Maybe they were following the UNIX philosophy of text-oriented design instead of binary, encrypted formats? ;)

              1. 3

                I know you’re saying it as a joke but the UNIX philosophy at this point is so chronically misunderstood that there’s a fair chance someone could read it and think “oh hey, good point”.

                If you… actually think that then I don’t know what to say other than, $ sudo cat /etc/shadow

                1. 1

                  Yeah, it’s a joke. They can always string together a pipeline of commands that do each step. Even the binary can be represented in a text format.

              2. 2

                Maybe they should start asking questions about how not to accidentally log the credentials at the scale of a small nation-state, instead of whiteboarding random programmer trivia.

                1. 2

                  Interesting, because this seems very similar to the first GDPR fine with knuddels.de. In contrast, Facebook did not leak the data though.

                  1. -8