remove non-UTF-8 characters from xml with declared encoding=utf-8 - Java
1) I get xml as java String with £ in it (I don't have access to interface right now, but I probably get xml as a java String). Can I use replaceAll(£, "") to get rid of this character?
I am assuming that you rather mean that you want to get rid of non-ASCII characters, because you're talking about a "legacy" side. You can get rid of anything outside the printable ASCII range using the following regex:
string = string.replaceAll("[^\\x20-\\x7e]", "");
2) I get xml as an array of bytes - how to handle this operation safely in that case?
You need to wrap the byte[]
in an ByteArrayInputStream
, so that you can read them in an UTF-8 encoded character stream using InputStreamReader
wherein you specify the encoding and then use a BufferedReader
to read it line by line.
E.g.
BufferedReader reader = null;try { reader = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(bytes), "UTF-8")); for (String line; (line = reader.readLine()) != null;) { line = line.replaceAll("[^\\x20-\\x7e]", ""); // ... } // ...
UTF-8 is an encoding; Unicode is a character set. But the GBP symbol is most definitely in the Unicode character set and therefore most certainly representable in UTF-8.
If you do in fact mean UTF-8, and you are actually trying to remove byte sequences that are not the valid encoding of a character in UTF-8, then...
CharsetDecoder utf8Decoder = Charset.forName("UTF-8").newDecoder();utf8Decoder.onMalformedInput(CodingErrorAction.IGNORE);utf8Decoder.onUnmappableCharacter(CodingErrorAction.IGNORE);ByteBuffer bytes = ...;CharBuffer parsed = utf8Decoder.decode(bytes);...