jq

The jq manual is good but long and omits some nuances. The wiki has more in-depth information but is even bigger and less structured (the jq Language Description might be interesting though). This document tries to be concise and exhaustive.

TODO:

• https://github.com/01mf02/jaq#basics
• https://rosettacode.org/wiki/Category:Jq

Comparison to awk

jq is for JSON what awk is for text, and fits well into a Unix-like environment.

Similarities:

• Takes data on stdin, produces data on stdout.
• Takes the program as a string argument or as a file with the -f option.
• Often used with one-liners but supports longer programs.
• Supports shebangs.

Differences:

• awk operates on newline-separated lines of text, jq operates on whitespace-separated JSON values.
• awk is mostly imperative, jq is mostly functional.

Basics

• A value has one of the following (JSON) types:
  • Null: null.
  • Boolean: true, false.
  • Number: 1.23.
  • String: "hello".
  • Array: [null, 1.23, true, "hello"].
  • Object: { "a": 1.23, "b": true, "c": "hello" }.
• A stream is (serialized as) values separated by (any) whitespace: null 1 true "hello".
• A filter f takes one(!) input stream and produces one(!) output stream.
• A pipe | takes two filters f1 and f2 and produces a filter f1 | f2. The input stream of the pipe filter is fed to f1, the output stream of f1 is fed to the input stream of f2, and the output stream of f2 is the output stream of the filter pipe.
• (In the typical case) the jq executable takes a filter as its only argument, the input stream of which is stdin and which output stream goes to stdout (with values pretty printed and separated by a single newline).

Filters

TODO: As seen above, a pipe is a filter and should be listed here instead of above.

One way to categorize filters is now many outputs they produce per input.

1 to 1

Identities

Passes the values in the input stream to the output stream unchanged.

• Identity: .. Does nothing else.
• Variable: f as $var. Passes the input stream to f and stores its output stream in the variable $var.
• Function: def func: f;. Defines func as an alias to filter f (which is usually a pipe).

Discarders

• Value literals. E.g. for every value in the input stream the filter 1 discards it and produces the number 1.

• The implicit filter in jq is the . identity filter

  $ printf '1 "hello" true' | jq
  1
  "hello"
  true

  $ echo '1 2 3' | jq '.'
  2
  3
  4

  $ echo '1 2 3' | jq '. + 1 | . * .'
  4
  9
  16

• jq has two basic components:
  • Noun:
  • Verb: A filter, takes an input stream and produces an output stream.

• If it seems like a filter takes two input streams, it is really copying the one input to the two places it’s used.