How do I do a case-insensitive string comparison?
Comparing strings in a case insensitive way seems trivial, but it's not. I will be using Python 3, since Python 2 is underdeveloped here.
The first thing to note is that case-removing conversions in Unicode aren't trivial. There is text for which
text.lower() != text.upper().lower(), such as
"ß".lower()#>>> 'ß'"ß".upper().lower()#>>> 'ss'
But let's say you wanted to caselessly compare
"Buße". Heck, you probably also want to compare
"BUẞE" equal - that's the newer capital form. The recommended way is to use
Return a casefolded copy of the string. Casefolded strings may be used for caseless matching.
Casefolding is similar to lowercasing but more aggressive because it is intended to remove all case distinctions in a string. [...]
Do not just use
casefold is not available, doing
.upper().lower() helps (but only somewhat).
Then you should consider accents. If your font renderer is good, you probably think
"ê" == "ê" - but it doesn't:
"ê" == "ê"#>>> False
This is because the accent on the latter is a combining character.
import unicodedata[unicodedata.name(char) for char in "ê"]#>>> ['LATIN SMALL LETTER E WITH CIRCUMFLEX'][unicodedata.name(char) for char in "ê"]#>>> ['LATIN SMALL LETTER E', 'COMBINING CIRCUMFLEX ACCENT']
unicodedata.normalize("NFKD", "ê") == unicodedata.normalize("NFKD", "ê")#>>> True
To finish up, here this is expressed in functions:
import unicodedatadef normalize_caseless(text): return unicodedata.normalize("NFKD", text.casefold())def caseless_equal(left, right): return normalize_caseless(left) == normalize_caseless(right)
Using Python 2, calling
.lower() on each string or Unicode object...
string1.lower() == string2.lower()
...will work most of the time, but indeed doesn't work in the situations @tchrist has described.
Assume we have a file called
unicode.txt containing the two strings
ΣΊΣΥΦΟΣ. With Python 2:
open("unicode.txt", 'r').read()print repr(utf8_bytes)'\xce\xa3\xce\xaf\xcf\x83\xcf\x85\xcf\x86\xce\xbf\xcf\x82\n\xce\xa3\xce\x8a\xce\xa3\xce\xa5\xce\xa6\xce\x9f\xce\xa3\n'u = utf8_bytes.decode('utf8')print uΣίσυφοςΣΊΣΥΦΟΣ first, second = u.splitlines()print first.lower()σίσυφοςprint second.lower()σίσυφοσ first.lower() == second.lower()Falsefirst.upper() == second.upper()Trueutf8_bytes =
The Σ character has two lowercase forms, ς and σ, and
.lower() won't help compare them case-insensitively.
However, as of Python 3, all three forms will resolve to ς, and calling lower() on both strings will work correctly:
open('unicode.txt', encoding='utf8').read()print(s)ΣίσυφοςΣΊΣΥΦΟΣ first, second = s.splitlines()print(first.lower())σίσυφοςprint(second.lower())σίσυφος first.lower() == second.lower()Truefirst.upper() == second.upper()Trues =
So if you care about edge-cases like the three sigmas in Greek, use Python 3.
(For reference, Python 2.7.3 and Python 3.3.0b1 are shown in the interpreter printouts above.)