team foundation server custom hosted build controller for selenium tests
I ended up moving away from the TFS hosted build controllers since you can't customize them.
I was able to create an azure VM with windows 2012 server installed on it. I installed TFS server and was able to link my TFS account to my on-premise TFS build controller.
Since I am using the azure VM, I am able to customize it, so I was able to download all my browsers, install Selenium IE driver and get the builds to work.
I am still working on how to automate the starting of the VM and taking it down when not in use so I don't get charged for compute hours when it's not in use, but it is nice having it always up so that it can automatically build after a code check-in.