Will you probably consider to use RequireJS. And add a fallback method if maps couldn't be loaded?
Time ago I created a service for my personal project. check it maybe can help you.
https://github.com/SaulBurgos/angularjs-module-googleMaps