1. 9
  1.  

  2. 4

    I’m curious if anyone is knowledgeable about how ad blockers and software like Brave browser fit in with the terms of use situation. Seems like if you are crawling and scraping for your own personal use, and not re-publishing, you might be able to craft your crawler/scraper to adhere as closely to TOU as ad blocking does.

    1. 3

      Brave is in a very precarious spot I think because they’re taking the content, remixing it, and showing it. That’s close to what aereo was doing. Actually probably more infringing than aereo. Maybe you can do it for yourself, personally, but it’s treacherous ground for a business model.

      1. 2

        I’m curious if anyone is knowledgeable about how ad blockers and software like Brave browser fit in with the terms of use situation.

        Personally, I don’t know. It’s a different topic.

        But if you consider that there are still a lot of grey areas in law about scraping/crawling, there are probably also a lot of grey areas about Ad Blockers. I’ve just googled it and I found that some German publishers sued Adblock Plus in the past. Not sure what happened to the other ad blockers.

        Seems like if you are crawling and scraping for your own personal use, and not re-publishing, you might be able to craft your crawler/scraper to adhere as closely to TOU as ad blocking does.

        I don’t think so. Because ToS/ToU often prohibit automatic data collection.

        1. 2

          But if you consider that there are still a lot of grey areas in law about scraping/crawling, there are probably also a lot of grey areas about Ad Blockers. I’ve just googled it and I found that some German publishers sued Adblock Plus in the past. Not sure what happened to the other ad blockers.

          AdBlock Plus has an “acceptable ads” product, which charges larger publishers a fee to be included on that list.

          https://adblockplus.org/acceptable-ads#revenue

          Springer sued AdBlock Plus and ad blocking itself was deemed legal, “acceptable ads” not.

        2. 1

          There have been plenty of attempts by publishers to sue adblockers with arguments along those lines. From what I’m aware they always lost.

          1. 1

            Apparently, Google and other big names attempted to sue Adblock Plus. But I don’t know how it turned out either.

            It would be interesting to do a bit more research on this topic. What we’d find out would probably be super interesting :)

        3. 1

          Great write up! I, for one, can’t wait (sense my sarcasm here) until we have cookie / evercookie/ IP based TOS CAPTCHAs to prove the TOS was agreed to before proceeding. And not one of these JavaScript based things…No! that’d be too easy to ignore or work around. I’m talking a kill bots style intrusion that happens way down low in the stack that can’t be circumvented and must be done to unlock the content. At the first sign of bot like behavior at a given IP, it’s another TOS CAPTCHA.

          1. 1

            Thanks for your kind remarks.

            And yep, what you suggest would be super effective! hehe :)

          2. 1

            Great article. This is something I think way too few people know. I used to be in that business myself, even though it was very local and so only nation wide (I am not living in the US) applied. It was really insightful.

            Funny side note: robots.txt was not binding, calls or emails of people saying you need to stop were. That’s how weird laws can be. Again, not the US. ;)

            Of course one should still respect it to not get a call. And another hint to not get into troubles: Sell the fact that you are crawling them! Backlinks, free visitors, etc.

            And one more thing. Crawling, scraping, filling out forms, sending post vs get requests, etc. can all be viewed differently by the law and a lot of common terms can mean different things. So ignore their technical meanings when dealing with the law. Make sure to learn what those things mean to lawyers. They can be funny and things that you never even considered. That’s by the way something one should do in general: Question each and every technical term!

            Here is the only thing I am not so sure about:

            “It’s the same as what my browser already does! Scraping a site is not technically different from using a web browser. I could gather data manually, anyway!”

            False. Terms of Service (ToS) often contain clauses that prohibit crawling/scraping/harvesting and automated uses of their associated services. You’re legally bound by those terms; it doesn’t matter that you could get that data manually.

            AFAIK ToS only apply if you register/explicitly agree. Is that true?

            Else it would be really weird. One would essentially create a law, rather than terms. Or in other words you could make a link and there having a ToS saying you are not allowed to visit or that you will have to pay now.

            1. 1

              Hey, thanks for the great feedback! :)

              Sell the fact that you are crawling them! Backlinks, free visitors, etc.

              You’re absolutely right, and I should have mentioned it in my post!

              AFAIK ToS only apply if you register/explicitly agree. Is that true?

              In some cases, courts ruled that since the defendants were logically aware of the ToS (even though they hadn’t explicitly agreed to them), they were enforceable. Take a look at bullets #7 and #8 of the section “The typical counterarguments brought by people” in my post. Whether or not ToS are enforceable seems to depend on the context.

              Or in other words you could make a link and there having a ToS saying you are not allowed to visit or that you will have to pay now.

              True, and some people successful did it. Take a look at Internet Archive v. Suzanne Shell.