How to handle weirdly combined websocket messages?
My assumption is that you're working only with English/ASCII characters and something probably messed the stream. (NOTE:I am assuming), there are no special characters, if it's so, then I will suggest you pass the entire json string into this function:
function cleanString(input) { var output = ""; for (var i=0; i<input.length; i++) { if (input.charCodeAt(i) <= 127) { output += input.charAt(i); } } console.log(output);}//examplecleanString("�~�")You can make reference to How to remove invalid UTF-8 characters from a JavaScript string?
EDIT
From an article by Internet Engineering Task Force (IETF),
A common class of security problems arises when sending text data using the wrong encoding. This protocol specifies that messages with a Text data type (as opposed to Binary or other types) contain UTF-8- encoded data. Although the length is still indicated and applications implementing this protocol should use the length to determine where the frame actually ends, sending data in an improper
The "Payload data" is text data encoded as UTF-8. Note that a particular text frame might include a partial UTF-8 sequence; however, the whole message MUST contain valid UTF-8. Invalid UTF-8 in reassembled messages is handled as described in Handling Errors in UTF-8-Encoded Data, which states that When an endpoint is to interpret a byte stream as UTF-8 but finds that the byte stream is not, in fact, a valid UTF-8 stream, that endpoint MUST _Fail the WebSocket Connection_. This rule applies both during the opening handshake and during subsequent data exchange.
I really believe that you error (or functionality) is coming from the server side which combines your individual messages, so I will suggest come up with a logic of ensuring that all your characters MUST be converted from Unicode to ASCII by first encoding the characters as UTF-8. And you might also want to install npm install --save-optional utf-8-validate to efficiently check if a message contains valid UTF-8 as required by the spec.
You might also want to pass in an if condition to help you do some checks;
this.ws.on('message', (rawdata) => { if (message.type === 'utf8') { // accept only text }I hope this gets to help.
It seems your output is having some spaces, If you have any spaces or if you find any special characters please use Unicode to full fill them.
Here is the list of Unicode characters
This might help I think.
Those characters are known as "REPLACEMENT CHARACTER" - used to replace an unknown, unrecognized or unrepresentable character.
From: https://en.wikipedia.org/wiki/Specials_(Unicode_block)
The replacement character � (often a black diamond with a white question mark or an empty square box) is a symbol found in the Unicode standard at code point U+FFFD in the Specials table. It is used to indicate problems when a system is unable to render a stream of data to a correct symbol. It is usually seen when the data is invalid and does not match any character
Checking the section 8 of the WebSocket protocol Error Handling:
8.1. Handling Errors in UTF-8 from the Server
When a client is to interpret a byte stream as UTF-8 but finds that the byte stream is not in fact a valid UTF-8 stream, then any bytes or sequences of bytes that are not valid UTF-8 sequences MUST be interpreted as a U+FFFD REPLACEMENT CHARACTER.
8.2. Handling Errors in UTF-8 from the Client
When a server is to interpret a byte stream as UTF-8 but finds that the byte stream is not in fact a valid UTF-8 stream, behavior is undefined. A server could close the connection, convert invalid byte sequences to U+FFFD REPLACEMENT CHARACTERs, store the data verbatim, or perform application-specific processing. Subprotocols layered on the WebSocket protocol might define specific behavior for servers.
Depends on the implementation or library in use how to deal with this, for example from this post Implementing Web Socket servers with Node.js:
socket.ondata = function(d, start, end) { //var data = d.toString('utf8', start, end); var original_data = d.toString('utf8', start, end); var data = original_data.split('\ufffd')[0].slice(1); if (data == "kill") { socket.end(); } else { sys.puts(data); socket.write("\u0000", "binary"); socket.write(data, "utf8"); socket.write("\uffff", "binary"); }};In this case, if a � is found it will do:
var data = original_data.split('\ufffd')[0].slice(1);if (data == "kill") { socket.end();} Another thing that you could do is to update node to the latest stable, from this post OpenSSL and Breaking UTF-8 Change (fixed in Node v0.8.27 and v0.10.29):
As of these releases, if you try and pass a string with an unmatched surrogate pair, Node will replace that character with the unknown unicode character (U+FFFD). To preserve the old behavior set the environment variable NODE_INVALID_UTF8 to anything (even nothing). If the environment variable is present at all it will revert to the old behavior.