Why do browsers not send the Authentication header when the credentials are provided in the URL?
Yes it is about security, and it is about choice.
First of all, when you use curl to do the operation you mention above, curl defaults to and assumes that you want Basic authentication (unless you specify something else, and yes the name is truly "Basic").
If you ask curl to do HTTP auth with a single method and you provide the credentials, then it decides it can send the auth headers right away. If you instead ask it to do several auth types (which libcurl the underlying library can do), then it won't send any auth header in the first request, but it will instead send a non-auth request to see which methods the server wants - like the browsers do.
In the browsers case, they always have the multiple-auth schemes approach so they don't assume you want Basic auth and by doing so, they don't blindly send your user+password over the network in the clear. They will then only do that if the server actually only wants Basic (or the browser doesn't support whatever more auth types the server supports).
I hope this clarifies things somewhat.