How can I escape characters in SQLite via bash shell?

The trouble with MarkusQ's solution is knowing which characters are special inside double quotes - there are quite a lot of them, including back-ticks, dollar-open parenthesis, dollar-variable, etc.

I would suggest it is better to enclose the string inside single quotes; then, each single quote inside the string needs to be replaced by the sequence quote, backslash, quote, quote:

sqlite3.bin contacts.db 'select * from contacts      where source = "Nancy'\''s notes"'

The first quote in the replacement terminates the current single-quoted string; the backslash-quote represents a literal single quote, and the final quote starts a new single-quoted string. Further, this works with Bourne, Korn, Bash and POSIX shells in general. (C Shell and derivatives have more complex rules needing backslashes to escape newlines, and so on.)

If bash is your only problem, enclose the whole thing in double quotes and then escape anything that's special within bash double quotes with a single backslash. E.g.:

sqlite3.bin contacts.db "select * from contacts where source = \"Nancy's notes on making \$\$\$\""

Here I use two single quotes that sqlite interprets as one.

sqlite3.bin contacts.db "select * from contacts where source = 'Nancy''s notes on making \$\$\$'"