1. 26

  2. 2

    Peloton uses the same idea, but it is built in to the database. Andy Pavlo has an entertaining talk about it. https://youtube.com/watch?v=mzMnyYdO8jk

    1. 2

      Excellent idea and smart implementation.

      1. 1

        I didn’t know Dexter could index our databases. Do I e-mail him queries or just pester him at the next BSD conference…?

        1. 1

          If I logged every statement to the database, I’d generate >30GB of logs per day. Does this scale to that level?

          1. 3

            It uses fingerprinting to group queries. Queries with the same parse tree but different values are grouped together.

            While dexter would have to ingest 30GB, the output would likely be much smaller if most queries can be grouped.

            1. 1

              it is likely that a lot of statement are duplicate no ?

              1. 1

                Certainly, though I doubt that helps Dexter scale much. The intake method appears to be “stream Postgres log files directly to Dexter”, so I’m just wondering if Dexter will be able to handle that.

              2. 1

                I’d look into replacing that first step with one based on pg_stat_statements.