Ruby 2.0.0 String#Match ArgumentError: invalid byte sequence in UTF-8
In Ruby 2.0 the encode
method is a no-op when encoding a string to its current encoding:
Please note that conversion from an encoding
enc
to the same encodingenc
is a no-op, i.e. the receiver is returned without any changes, and no exceptions are raised, even if there are invalid bytes.
This changed in 2.1, which also added the scrub
method as an easier way to do this.
If you are unable to upgrade to 2.1, you’ll have to encode into a different encoding and back in order to remove invalid bytes, something like:
if ! s.valid_encoding? s = s.encode("UTF-16be", :invalid=>:replace, :replace=>"?").encode('UTF-8')end
Since you're using Rails and not just Ruby you can also use tidy_bytes. This works with Ruby 2.0 and also will probably give you back sensible data instead of just replacement characters.