How to urlencode data for curl command?

Use curl --data-urlencode; from man curl:

This posts data, similar to the other --data options with the exception that this performs URL-encoding. To be CGI-compliant, the <data> part should begin with a name followed by a separator and a content specification.

Example usage:

curl \    --data-urlencode "paramName=value" \    --data-urlencode "secondParam=value" \    http://example.com

See the man page for more info.

This requires curl 7.18.0 or newer (released January 2008). Use curl -V to check which version you have.

You can as well encode the query string:

curl -G \    --data-urlencode "p1=value 1" \    --data-urlencode "p2=value 2" \    http://example.com    # http://example.com?p1=value%201&p2=value%202

Here is the pure BASH answer.

rawurlencode() {  local string="${1}"  local strlen=${#string}  local encoded=""  local pos c o  for (( pos=0 ; pos<strlen ; pos++ )); do     c=${string:$pos:1}     case "$c" in        [-_.~a-zA-Z0-9] ) o="${c}" ;;        * )               printf -v o '%%%02x' "'$c"     esac     encoded+="${o}"  done  echo "${encoded}"    # You can either set a return variable (FASTER)   REPLY="${encoded}"   #+or echo the result (EASIER)... or both... :p}

You can use it in two ways:

easier:  echo http://url/q?=$( rawurlencode "$args" )faster:  rawurlencode "$args"; echo http://url/q?${REPLY}

[edited]

Here's the matching rawurldecode() function, which - with all modesty - is awesome.

# Returns a string in which the sequences with percent (%) signs followed by# two hex digits have been replaced with literal characters.rawurldecode() {  # This is perhaps a risky gambit, but since all escape characters must be  # encoded, we can replace %NN with \xNN and pass the lot to printf -b, which  # will decode hex for us  printf -v REPLY '%b' "${1//%/\\x}" # You can either set a return variable (FASTER)  echo "${REPLY}"  #+or echo the result (EASIER)... or both... :p}

With the matching set, we can now perform some simple tests:

$ diff rawurlencode.inc.sh \        <( rawurldecode "$( rawurlencode "$( cat rawurlencode.inc.sh )" )" ) \        && echo MatchedOutput: Matched

And if you really really feel that you need an external tool (well, it will go a lot faster, and might do binary files and such...) I found this on my OpenWRT router...

replace_value=$(echo $replace_value | sed -f /usr/lib/ddns/url_escape.sed)

Where url_escape.sed was a file that contained these rules:

# sed url escapings:%:%25:gs: :%20:gs:<:%3C:gs:>:%3E:gs:#:%23:gs:{:%7B:gs:}:%7D:gs:|:%7C:gs:\\:%5C:gs:\^:%5E:gs:~:%7E:gs:\[:%5B:gs:\]:%5D:gs:`:%60:gs:;:%3B:gs:/:%2F:gs:?:%3F:gs^:^%3A^gs:@:%40:gs:=:%3D:gs:&:%26:gs:\$:%24:gs:\!:%21:gs:\*:%2A:g

Another option is to use jq:

$ printf %s 'encode this'|jq -sRr @uriencode%20this$ jq -rn --arg x 'encode this' '$x|@uri'encode%20this

-r (--raw-output) outputs the raw contents of strings instead of JSON string literals. -n (--null-input) doesn't read input from STDIN.

-R (--raw-input) treats input lines as strings instead of parsing them as JSON, and -sR (--slurp --raw-input) reads the input into a single string. You can replace -sRr with -Rr if your input only contains a single line, or if you don't want to replace linefeeds with %0A:

$ printf %s\\n 'multiple lines' 'of text'|jq -Rr @urimultiple%20linesof%20text$ printf %s\\n 'multiple lines' 'of text'|jq -sRr @urimultiple%20lines%0Aof%20text%0A

Or this percent-encodes all bytes:

xxd -p|tr -d \\n|sed 's/../%&/g'