Best way to replace multiple characters in a string?
Replacing two characters
I timed all the methods in the current answers along with one extra.
With an input string of abc&def#ghi
and replacing & -> \& and # -> \#, the fastest way was to chain together the replacements like this: text.replace('&', '\&').replace('#', '\#')
.
Timings for each function:
- a) 1000000 loops, best of 3: 1.47 μs per loop
- b) 1000000 loops, best of 3: 1.51 μs per loop
- c) 100000 loops, best of 3: 12.3 μs per loop
- d) 100000 loops, best of 3: 12 μs per loop
- e) 100000 loops, best of 3: 3.27 μs per loop
- f) 1000000 loops, best of 3: 0.817 μs per loop
- g) 100000 loops, best of 3: 3.64 μs per loop
- h) 1000000 loops, best of 3: 0.927 μs per loop
- i) 1000000 loops, best of 3: 0.814 μs per loop
Here are the functions:
def a(text): chars = "&#" for c in chars: text = text.replace(c, "\\" + c)def b(text): for ch in ['&','#']: if ch in text: text = text.replace(ch,"\\"+ch)import redef c(text): rx = re.compile('([&#])') text = rx.sub(r'\\\1', text)RX = re.compile('([&#])')def d(text): text = RX.sub(r'\\\1', text)def mk_esc(esc_chars): return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])esc = mk_esc('&#')def e(text): esc(text)def f(text): text = text.replace('&', '\&').replace('#', '\#')def g(text): replacements = {"&": "\&", "#": "\#"} text = "".join([replacements.get(c, c) for c in text])def h(text): text = text.replace('&', r'\&') text = text.replace('#', r'\#')def i(text): text = text.replace('&', r'\&').replace('#', r'\#')
Timed like this:
python -mtimeit -s"import time_functions" "time_functions.a('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.b('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.c('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.d('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.e('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.f('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.g('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.h('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.i('abc&def#ghi')"
Replacing 17 characters
Here's similar code to do the same but with more characters to escape (\`*_{}>#+-.!$):
def a(text): chars = "\\`*_{}[]()>#+-.!$" for c in chars: text = text.replace(c, "\\" + c)def b(text): for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']: if ch in text: text = text.replace(ch,"\\"+ch)import redef c(text): rx = re.compile('([&#])') text = rx.sub(r'\\\1', text)RX = re.compile('([\\`*_{}[]()>#+-.!$])')def d(text): text = RX.sub(r'\\\1', text)def mk_esc(esc_chars): return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])esc = mk_esc('\\`*_{}[]()>#+-.!$')def e(text): esc(text)def f(text): text = text.replace('\\', '\\\\').replace('`', '\`').replace('*', '\*').replace('_', '\_').replace('{', '\{').replace('}', '\}').replace('[', '\[').replace(']', '\]').replace('(', '\(').replace(')', '\)').replace('>', '\>').replace('#', '\#').replace('+', '\+').replace('-', '\-').replace('.', '\.').replace('!', '\!').replace('$', '\$')def g(text): replacements = { "\\": "\\\\", "`": "\`", "*": "\*", "_": "\_", "{": "\{", "}": "\}", "[": "\[", "]": "\]", "(": "\(", ")": "\)", ">": "\>", "#": "\#", "+": "\+", "-": "\-", ".": "\.", "!": "\!", "$": "\$", } text = "".join([replacements.get(c, c) for c in text])def h(text): text = text.replace('\\', r'\\') text = text.replace('`', r'\`') text = text.replace('*', r'\*') text = text.replace('_', r'\_') text = text.replace('{', r'\{') text = text.replace('}', r'\}') text = text.replace('[', r'\[') text = text.replace(']', r'\]') text = text.replace('(', r'\(') text = text.replace(')', r'\)') text = text.replace('>', r'\>') text = text.replace('#', r'\#') text = text.replace('+', r'\+') text = text.replace('-', r'\-') text = text.replace('.', r'\.') text = text.replace('!', r'\!') text = text.replace('$', r'\$')def i(text): text = text.replace('\\', r'\\').replace('`', r'\`').replace('*', r'\*').replace('_', r'\_').replace('{', r'\{').replace('}', r'\}').replace('[', r'\[').replace(']', r'\]').replace('(', r'\(').replace(')', r'\)').replace('>', r'\>').replace('#', r'\#').replace('+', r'\+').replace('-', r'\-').replace('.', r'\.').replace('!', r'\!').replace('$', r'\$')
Here's the results for the same input string abc&def#ghi
:
- a) 100000 loops, best of 3: 6.72 μs per loop
- b) 100000 loops, best of 3: 2.64 μs per loop
- c) 100000 loops, best of 3: 11.9 μs per loop
- d) 100000 loops, best of 3: 4.92 μs per loop
- e) 100000 loops, best of 3: 2.96 μs per loop
- f) 100000 loops, best of 3: 4.29 μs per loop
- g) 100000 loops, best of 3: 4.68 μs per loop
- h) 100000 loops, best of 3: 4.73 μs per loop
- i) 100000 loops, best of 3: 4.24 μs per loop
And with a longer input string (## *Something* and [another] thing in a longer sentence with {more} things to replace$
):
- a) 100000 loops, best of 3: 7.59 μs per loop
- b) 100000 loops, best of 3: 6.54 μs per loop
- c) 100000 loops, best of 3: 16.9 μs per loop
- d) 100000 loops, best of 3: 7.29 μs per loop
- e) 100000 loops, best of 3: 12.2 μs per loop
- f) 100000 loops, best of 3: 5.38 μs per loop
- g) 10000 loops, best of 3: 21.7 μs per loop
- h) 100000 loops, best of 3: 5.7 μs per loop
- i) 100000 loops, best of 3: 5.13 μs per loop
Adding a couple of variants:
def ab(text): for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']: text = text.replace(ch,"\\"+ch)def ba(text): chars = "\\`*_{}[]()>#+-.!$" for c in chars: if c in text: text = text.replace(c, "\\" + c)
With the shorter input:
- ab) 100000 loops, best of 3: 7.05 μs per loop
- ba) 100000 loops, best of 3: 2.4 μs per loop
With the longer input:
- ab) 100000 loops, best of 3: 7.71 μs per loop
- ba) 100000 loops, best of 3: 6.08 μs per loop
So I'm going to use ba
for readability and speed.
Addendum
Prompted by haccks in the comments, one difference between ab
and ba
is the if c in text:
check. Let's test them against two more variants:
def ab_with_check(text): for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']: if ch in text: text = text.replace(ch,"\\"+ch)def ba_without_check(text): chars = "\\`*_{}[]()>#+-.!$" for c in chars: text = text.replace(c, "\\" + c)
Times in μs per loop on Python 2.7.14 and 3.6.3, and on a different machine from the earlier set, so cannot be compared directly.
╭────────────╥──────┬───────────────┬──────┬──────────────────╮│ Py, input ║ ab │ ab_with_check │ ba │ ba_without_check │╞════════════╬══════╪═══════════════╪══════╪══════════════════╡│ Py2, short ║ 8.81 │ 4.22 │ 3.45 │ 8.01 ││ Py3, short ║ 5.54 │ 1.34 │ 1.46 │ 5.34 │├────────────╫──────┼───────────────┼──────┼──────────────────┤│ Py2, long ║ 9.3 │ 7.15 │ 6.85 │ 8.55 ││ Py3, long ║ 7.43 │ 4.38 │ 4.41 │ 7.02 │└────────────╨──────┴───────────────┴──────┴──────────────────┘
We can conclude that:
Those with the check are up to 4x faster than those without the check
ab_with_check
is slightly in the lead on Python 3, butba
(with check) has a greater lead on Python 2However, the biggest lesson here is Python 3 is up to 3x faster than Python 2! There's not a huge difference between the slowest on Python 3 and fastest on Python 2!
Here is a python3 method using str.translate
and str.maketrans
:
s = "abc&def#ghi"print(s.translate(str.maketrans({'&': '\&', '#': '\#'})))
The printed string is abc\&def\#ghi
.