MediaSource error: This SourceBuffer has been removed from the parent media source

Ultimately the issue was that I was sending h264 video down the websocket. The MediaSource API only supports MPEG-DASH and VP8 with keyframed segments currently (on Chrome 35).

Additionally, once I tried VP8, I saw that I was adding some frames out of order.

• Adding if (buffer.updating || queue.length > 0) in websocket.onmessage was required.
• Adding if (queue.length > 0 && !buffer.updating) in buffer.addEventListener('update', ...) was also required.

Note: I applied the edits mentioned here to the code in the question, so the only issue with the code in the question is that the codec is wrong

Chrome is unbelievably picky when it comes to SourceBuffer codecs. Worse, it will return useless and misleading error messages as in case of the OP.

It looks like the MediaSource is closing immediately after the call to buffer.appendData()

That's exactly the case: Chrome is not happy with the video as it somehow doesn't exactly conform to its likings and simply closes the source buffer without notice.

Possible solutions to look for:

1. Transcode your MP4 files using ffmpeg, explained here (headline Fragmenting).
2. Use mp4file of mp4v2 to find out the exact MP4 codec, explained here and here.
3. If you don't have audio, omit the audio codec part: e.g. 'video/mp4; codecs="avc1.64001F"' instead of 'video/mp4; codecs="avc1.64001F, mp4a.40.2"'.