1. 23

  2. 5

    Hah, that’s a fun one.

    We had a similar one, our logging was originally using NCSA Common log format; for almost a year we would be seemingly randomly seeing GET requests to URLs that we expected POSTs to, which would result in 405 errors. For example if we have POST https://lobste.rs/write-data/unique-guid-000-00000, we would then see a GET https://lobste.rs/write-data/unique-guid-000-00000 at a later time. These errors would appear MUCH more frequently when we were already dealing with problems, and we’d see it almost always when debugging something. We had a few little quick instances of brainstorming where they were coming from but never could figure it out with the tools & data we had. We just assumed things were haunted, it was never that big of a deal because the traffic was low, and it never was related to the issues we were debugging.

    Finally we replaced our common log format with structured json log lines which contained more data… It was Slack. Every time we were debugging something, people were posting log snippits which had the URLs in them, and slack was trying to unfurl them. Which explained why it was only realllly showing up on the unique calls we were debugging.

    1. 1

      Reminds me of when skype claimed to be so secure/private. You dropped a link in the chat, and some microsoft IP came visiting your server. (Note that there was no unfurling AFAIK.)

    2. 2

      Stripe team acknowledged the issue


      Will be interesting to see how long it takes to get fixed

      1. 2

        Annoyingly, blocking these requests did not surface any error – it seems that whatever that Stripe is scraping is non-essential

        Seems like a lot of the value add for Stripe is their magic sauce differentiating fraud from non-fraud. And, much like people, they try to widen their “line-of-sight” on their counter party. I assume they do this both to check for red flags, and probably also to check for “green” flags too. Stripe scraping the website might just be part of their fraud differentiation, and that seems to me to be consistent with the blocked scraping not surfacing an error.

        1. 1

          Extremely advanced anti-fraud systems are fairy standard across the American payment card industry, from what I’ve heard. As a developer, Stripe’s SDK and API, especially around subscription management and comprehensive story for suspended payments, is the primary reason to choose them.

          1. 1

            They really don’t care as much about fraud as we be optimal for society, because their customers, the ones receiving the fraudulent payments, are the ones that bear the risk. And those customers don’t have much choice due to the creditcard duopoly.

          2. 1

            It being anti fraud would also explain why it claims to be Chrome rather than explicitly saying “Stripe” in the user agent string.

            It’s slightly worrying that it says outdated Chrome. In the unlikely but not totally implausible event that they really are using headless Chrome to scrape pages, they could have all sorts of unpatched old vulnerabilities. ;)

            1. 5

              I tinkered a bit more with it after posting this, specifically because I was wondering the same thing, and yeah… it really looks like they are using a very old version of headless Chrome.

              1. 2

                So we’re finally at the point where you don’t target normal visitors, but towards scarper bots.

                1. 1

                  Oh eek

                  I hope they sandbox that well

            2. 1

              This reminds me of an issue I have with Discord and gmail. Every time I get a Discord invite to my gmail account the link is invalid. Is it possible that Google is accessing my link before me and ruining the invite?

              1. 1

                I’m unable to replicate this with an invite sent from outlook to gmail. Is it possible that the invites are expiring? How are the invites sourced?

                Edit: Actually, merely viewing the invite isn’t enough to decrease the number of uses on a limited-use invite

                1. 1

                  They are expiring within 2 minutes of generation. Weird. Thanks for thinking about it.