componentDidCatch + window.addEventListener('error', cb) don't behave as expected
Seems like this is just a development behaviour. Please take a look at this comment
In development environment, React uses a trick: caught exceptions are thrown inside a fake DOM event which makes them reported by window.onerror, but then React actually catches them so that they don't propagate up (in case there is an error boundary — otherwise they are rethrown).
In production errors caught by error boundaries stay caught.
I've just tested your example in production and alert does not show when error is thrown in render method.