Escaping characters in bash (for JSON)
Using Python:
This solution is not pure bash, but it's non-invasive and handles unicode.
json_escape () { printf '%s' "$1" | python -c 'import json,sys; print(json.dumps(sys.stdin.read()))'}
Note that JSON is part of the standard python libraries and has been for a long time, so this is a pretty minimal python dependency.
Or using PHP:
json_escape () { printf '%s' "$1" | php -r 'echo json_encode(file_get_contents("php://stdin"));'}
Use like so:
$ json_escape "ヤホー""\u30e4\u30db\u30fc"
jq
can do this.
Lightweight, free, and written in C, jq
enjoys widespread community support with over 15k stars on GitHub. I personally find it very speedy and useful in my daily workflow.
Convert string to JSON
$ echo '猫に小判' | jq -aRs ."\u732b\u306b\u5c0f\u5224\n"
$ printf 'ô\nè\nà\n' | jq -Rs ."ô\nè\nà\n"
To explain,
-a
means "ascii output" (omitted in the second example)-R
means "raw input"-s
means "include linebreaks".
means "output the root of the JSON document"
Git + Grep Use Case
To fix the code example given by the OP, simply pipe through jq.
MSG=`git log -n 1 --format=oneline | grep -o ' .\+' | jq -aRs .`
Instead of worrying about how to properly quote the data, just save it to a file and use the @
construct that curl
allows with the --data
option. To ensure that the output of git
is correctly escaped for use as a JSON value, use a tool like jq
to generate the JSON, instead of creating it manually.
jq -n --arg msg "$(git log -n 1 --format=oneline | grep -o ' .\+')" \ '{payload: { message: $msg }}' > git-tmp.txtcurl -i -X POST \ -H 'Accept: application/text' \ -H 'Content-type: application/json' \ -d @git-tmp.txt \ 'https://example.com'
You can also read directly from standard input using -d @-
; I leave that as an exercise for the reader to construct the pipeline that reads from git
and produces the correct payload message to upload with curl
.
(Hint: it's jq ... | curl ... -d@- 'https://example.com'
)