Best way to replace multiple characters in a string? Best way to replace multiple characters in a string? python python

Best way to replace multiple characters in a string?


Replacing two characters

I timed all the methods in the current answers along with one extra.

With an input string of abc&def#ghi and replacing & -> \& and # -> \#, the fastest way was to chain together the replacements like this: text.replace('&', '\&').replace('#', '\#').

Timings for each function:

  • a) 1000000 loops, best of 3: 1.47 μs per loop
  • b) 1000000 loops, best of 3: 1.51 μs per loop
  • c) 100000 loops, best of 3: 12.3 μs per loop
  • d) 100000 loops, best of 3: 12 μs per loop
  • e) 100000 loops, best of 3: 3.27 μs per loop
  • f) 1000000 loops, best of 3: 0.817 μs per loop
  • g) 100000 loops, best of 3: 3.64 μs per loop
  • h) 1000000 loops, best of 3: 0.927 μs per loop
  • i) 1000000 loops, best of 3: 0.814 μs per loop

Here are the functions:

def a(text):    chars = "&#"    for c in chars:        text = text.replace(c, "\\" + c)def b(text):    for ch in ['&','#']:        if ch in text:            text = text.replace(ch,"\\"+ch)import redef c(text):    rx = re.compile('([&#])')    text = rx.sub(r'\\\1', text)RX = re.compile('([&#])')def d(text):    text = RX.sub(r'\\\1', text)def mk_esc(esc_chars):    return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])esc = mk_esc('&#')def e(text):    esc(text)def f(text):    text = text.replace('&', '\&').replace('#', '\#')def g(text):    replacements = {"&": "\&", "#": "\#"}    text = "".join([replacements.get(c, c) for c in text])def h(text):    text = text.replace('&', r'\&')    text = text.replace('#', r'\#')def i(text):    text = text.replace('&', r'\&').replace('#', r'\#')

Timed like this:

python -mtimeit -s"import time_functions" "time_functions.a('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.b('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.c('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.d('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.e('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.f('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.g('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.h('abc&def#ghi')"python -mtimeit -s"import time_functions" "time_functions.i('abc&def#ghi')"

Replacing 17 characters

Here's similar code to do the same but with more characters to escape (\`*_{}>#+-.!$):

def a(text):    chars = "\\`*_{}[]()>#+-.!$"    for c in chars:        text = text.replace(c, "\\" + c)def b(text):    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:        if ch in text:            text = text.replace(ch,"\\"+ch)import redef c(text):    rx = re.compile('([&#])')    text = rx.sub(r'\\\1', text)RX = re.compile('([\\`*_{}[]()>#+-.!$])')def d(text):    text = RX.sub(r'\\\1', text)def mk_esc(esc_chars):    return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s])esc = mk_esc('\\`*_{}[]()>#+-.!$')def e(text):    esc(text)def f(text):    text = text.replace('\\', '\\\\').replace('`', '\`').replace('*', '\*').replace('_', '\_').replace('{', '\{').replace('}', '\}').replace('[', '\[').replace(']', '\]').replace('(', '\(').replace(')', '\)').replace('>', '\>').replace('#', '\#').replace('+', '\+').replace('-', '\-').replace('.', '\.').replace('!', '\!').replace('$', '\$')def g(text):    replacements = {        "\\": "\\\\",        "`": "\`",        "*": "\*",        "_": "\_",        "{": "\{",        "}": "\}",        "[": "\[",        "]": "\]",        "(": "\(",        ")": "\)",        ">": "\>",        "#": "\#",        "+": "\+",        "-": "\-",        ".": "\.",        "!": "\!",        "$": "\$",    }    text = "".join([replacements.get(c, c) for c in text])def h(text):    text = text.replace('\\', r'\\')    text = text.replace('`', r'\`')    text = text.replace('*', r'\*')    text = text.replace('_', r'\_')    text = text.replace('{', r'\{')    text = text.replace('}', r'\}')    text = text.replace('[', r'\[')    text = text.replace(']', r'\]')    text = text.replace('(', r'\(')    text = text.replace(')', r'\)')    text = text.replace('>', r'\>')    text = text.replace('#', r'\#')    text = text.replace('+', r'\+')    text = text.replace('-', r'\-')    text = text.replace('.', r'\.')    text = text.replace('!', r'\!')    text = text.replace('$', r'\$')def i(text):    text = text.replace('\\', r'\\').replace('`', r'\`').replace('*', r'\*').replace('_', r'\_').replace('{', r'\{').replace('}', r'\}').replace('[', r'\[').replace(']', r'\]').replace('(', r'\(').replace(')', r'\)').replace('>', r'\>').replace('#', r'\#').replace('+', r'\+').replace('-', r'\-').replace('.', r'\.').replace('!', r'\!').replace('$', r'\$')

Here's the results for the same input string abc&def#ghi:

  • a) 100000 loops, best of 3: 6.72 μs per loop
  • b) 100000 loops, best of 3: 2.64 μs per loop
  • c) 100000 loops, best of 3: 11.9 μs per loop
  • d) 100000 loops, best of 3: 4.92 μs per loop
  • e) 100000 loops, best of 3: 2.96 μs per loop
  • f) 100000 loops, best of 3: 4.29 μs per loop
  • g) 100000 loops, best of 3: 4.68 μs per loop
  • h) 100000 loops, best of 3: 4.73 μs per loop
  • i) 100000 loops, best of 3: 4.24 μs per loop

And with a longer input string (## *Something* and [another] thing in a longer sentence with {more} things to replace$):

  • a) 100000 loops, best of 3: 7.59 μs per loop
  • b) 100000 loops, best of 3: 6.54 μs per loop
  • c) 100000 loops, best of 3: 16.9 μs per loop
  • d) 100000 loops, best of 3: 7.29 μs per loop
  • e) 100000 loops, best of 3: 12.2 μs per loop
  • f) 100000 loops, best of 3: 5.38 μs per loop
  • g) 10000 loops, best of 3: 21.7 μs per loop
  • h) 100000 loops, best of 3: 5.7 μs per loop
  • i) 100000 loops, best of 3: 5.13 μs per loop

Adding a couple of variants:

def ab(text):    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:        text = text.replace(ch,"\\"+ch)def ba(text):    chars = "\\`*_{}[]()>#+-.!$"    for c in chars:        if c in text:            text = text.replace(c, "\\" + c)

With the shorter input:

  • ab) 100000 loops, best of 3: 7.05 μs per loop
  • ba) 100000 loops, best of 3: 2.4 μs per loop

With the longer input:

  • ab) 100000 loops, best of 3: 7.71 μs per loop
  • ba) 100000 loops, best of 3: 6.08 μs per loop

So I'm going to use ba for readability and speed.

Addendum

Prompted by haccks in the comments, one difference between ab and ba is the if c in text: check. Let's test them against two more variants:

def ab_with_check(text):    for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:        if ch in text:            text = text.replace(ch,"\\"+ch)def ba_without_check(text):    chars = "\\`*_{}[]()>#+-.!$"    for c in chars:        text = text.replace(c, "\\" + c)

Times in μs per loop on Python 2.7.14 and 3.6.3, and on a different machine from the earlier set, so cannot be compared directly.

╭────────────╥──────┬───────────────┬──────┬──────────────────╮│ Py, input  ║  ab  │ ab_with_check │  ba  │ ba_without_check │╞════════════╬══════╪═══════════════╪══════╪══════════════════╡│ Py2, short ║ 8.814.223.458.01          ││ Py3, short ║ 5.541.341.465.34          │├────────────╫──────┼───────────────┼──────┼──────────────────┤│ Py2, long  ║ 9.37.156.858.55          ││ Py3, long  ║ 7.434.384.417.02          │└────────────╨──────┴───────────────┴──────┴──────────────────┘

We can conclude that:

  • Those with the check are up to 4x faster than those without the check

  • ab_with_check is slightly in the lead on Python 3, but ba (with check) has a greater lead on Python 2

  • However, the biggest lesson here is Python 3 is up to 3x faster than Python 2! There's not a huge difference between the slowest on Python 3 and fastest on Python 2!


>>> string="abc&def#ghi">>> for ch in ['&','#']:...   if ch in string:...      string=string.replace(ch,"\\"+ch)...>>> print stringabc\&def\#ghi


Here is a python3 method using str.translate and str.maketrans:

s = "abc&def#ghi"print(s.translate(str.maketrans({'&': '\&', '#': '\#'})))

The printed string is abc\&def\#ghi.


matomo