Reminds me of the textql utility which always seemed very cool to me.
Huh, I hadn’t seen textql before. Looks like it even uses SQLite under the hood!
See also: sqlite-utils, dsq (I develop), harelba/q, trdsql. General comparison here, benchmarks here.
This was extremely interesting, thanks . And It showed me there’s also a org Babel plug-in! https://github.com/fritzgrabo/ob-dsq
I’ve done this before with large datasets, and it’s really cool! I’ve also tried loading into a local Postgres instance to query large datasets. While both work, I found xsv a bit faster for my specific use case of simple row/column filtering on somewhat large CSV files. This was a while ago though, so it might not be the case now. Obviously which tool is better depends on what you’re trying to do with the data.
Has anybody benchmarked this against other options for similar results such as CSVKit’s csvsql or the Zed project’s zq query tool? Less SQL like options like the AWK variant that has direct CSV support and duckdb or others would be interesting to see benchmarked too.
There are links to benchmarks in this post. :)
Only 1 of the 4 I listed is benchmarked or even mentioned in any of the links I see.
By the way, with .mode csv and .import your-file.csv table_name you can also do it interactively like a normal sqlite table
.import your-file.csv table_name
Clever!! This beats awk by a long shot!