How do I detect requests initiated by the new fetch standard? How should I detect an AJAX request in general?
Check out this issue on the Github's polyfill repository, specially this comment.
Since the X-Requested-With header is not an standard, they are using a wrapper that provides some of the missing behavior.
If you need more guidance, check this lines of the wrapper code:
function headers(options) { options = options || {} options.headers = options.headers || {} options.headers['X-Requested-With'] = 'XMLHttpRequest' return options}
Why is a request identifying its initiator not enforced through standards
Because it shouldn't matter.
HTTP clients ask for the thing they want to get. The server should give it to them.
Giving clients something different based on what the client is tends to cause more problems than it solves. User-Agent sniffing being a prime example.
Should something else be used for decision-making (e.g. clients providing the content-type they expect in response)?
Yes.
The Accept header is specifically provided for allowing a client to specify which format they would prefer the data in when an HTTP resource is available in multiple formats.