Why can't I escape quote in gawk?

The idiom to do this is to create a variable which contains the single quote and then use that:

scanimage -L | gawk '/N650U/ {print gensub(q"`", "", "g", $2)}' q="'"

However, since you are using it in a character class, that is not going to work so you'll need to do this:

scanimage -L | gawk '/N650U/ {print gensub("[`'\'']", "", "g", $2)}'                    <--      1st pair       -->  <--   2nd pair  -->

Another alternative if using bash is to use $'' which does support escaping single-quotes

scanimage -L | gawk $'/N650U/ {print gensub("[`\']", "", "g", $2)}'

All you are doing in the 2nd case is creating a single-quote pair right before your literal single-quote, escaping the single quote so the shell doesn't interpret it and then make another single-quote pair after it.

Example with single-quote in a regex

$ echo $'foo`\'' | awk '{gsub(/[o`'\'']/,"#")}1'f####

Example with single-quote outside a regex

$ echo "foo" | awk '{print q$0q}' q="'"'foo'

Example with single-quote inside $''

echo $'foo`\'' | awk $'{gsub(/[o`\']/,"#")}1'f####

There's no special character in single quotes including backslash(\).

Enclosing characters in single quotes (') preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

You can change the command to:

$ scanimage -L | awk '/N650U/ {print gensub("['"'"'`]", "", "g", $2)}'

Shell '...' doesn't support backslash escapes. You'll have to use "..." instead, I'm afraid.

gawk "/N650U/ {print gensub(\"['`]\", \"\", \"g\", \$2)}\"

(Note that shell "..." does expand $ variables, so you need to escape that as well!)