I’m continuing my efforts to understand the basics of jq by writing posts that a past version of myself would have appreciated. I hope this sort of post helps folks to get their head around core jq mechanisms such as iterators, and also to become more comfortable with the shapes of JSON data that jq reads and writes.
I recently did something similar when processing the output of an AWS CloudWatch logs insights query using the AWS CLI and aws logs get-query-results, where the shape of the data is roughly:
aws logs get-query-results
aws logs get-query-results --region=us-east-1 --query-id $query_id > query-results.json
jq -r \
'[ .results |
map(with_entries(if .key == "field" then .key = "key" else . end)) |
from_entries ] |
(map(keys) | add | unique) as $cols |
map(. as $row | $cols | map($row[.])) as $rows | $cols, $rows |
@csv' query-results.json > query-results.csv
This is why I created https://fx.wtf, didn’t wanted to learn a new language =)
Got solution in under 1m:
fx entities.json '.value.map(x => Object.values(x).map(JSON.stringify)).join("\n")'
I do like fx, especially it’s “exploration editor” but also the fact that I can write in JS. But there’s something about jq’s language that draws me in.
Can you explain in bit more? Maybe I can improve something in fx)
Sorry, I wasn’t clear. It’s not that I see improvement requirements in fx, it’s that I’m attracted to jq because of the language. This is why I’m using jq rather than JS-in-fx to explore JSON, i.e. not for any negative reason.