1. 17
  1.  

  2. 7

    Common Crawl is the basis of most (or all?) of the index used by the Alexandria search engine. It’s also used by the Web Data Commons which extracts, publishes, and measures usage of structured data across the Web.

    It’s a great resource.

    1. 7

      This is an aside at best but how are we not yet past the point where we let people write “change the world” when what they mean is “sell something” or just “program the computer”

      1. 3

        Maybe because letting people choose their own words is part of the social norm. You’re welcome to complain about it, but let implied we have some say in the words they choose.

        1. 1

          Because who really cares that much about someone else’s hyperbole?