1. 9

  2. 2

    It is better to use bigquery’s mirror of Hacker News, than to query the endpoint millions of times. Here is an example of using that data in bigquery: https://hoffa.medium.com/hacker-news-on-bigquery-now-with-daily-updates-so-what-are-the-top-domains-963d3c68b2e2 But you can also get the whole dump from there, and then import it into whatever data warehouse you want.

    1. 2

      Nice, but I wanted to specifically go with Snowflake as my database rather than BigQuery. Good point that I could have used BigQuery to get the initial dump and then load it into Snowflake.