How does TestFlight do it?
This article showed how Apples OTA implementation works and can be used outside enterprises as well: ios wireless app distribution
The complete process is documented by Apple.Apple also published documentation and sample code for registering devices and get the UDID by using profiles, so your website can detect which device is calling.
Some additional solutions with different strenghts:
iOS Beta Builder, a Mac Application to create the website by using a build. Simply upload the resulting files to your webserver.
Diawi: Simple Web service. Upload your IPA file, optionally set a password and send a link to your testers.
AppSendr: Web service for beta build hosting, similar to Testflight, but does not include the device registration process. But provides deployment utilities to automatically upload new versions.
HockeyKit: Open source project for hosting beta versions on your own PHP5 server with additional functionalities like an client for In-App-Updates, automatic device specific web sites and handling multiple applications. Completely file and directory based.
HockeyApp: Web Service for beta build hosting, In-App-Updates, Statistics, and including device registration, invite and recruitment. Also provides server side crash report collection, symbolication (for all threads) and crash grouping for beta and app store apps (iOS + Mac). SDKs are open source, using HockeyKit, QuincyKit and PLCrashReporter (which is the only safe solution on how to do crash report collection on iOS, see this article.
Note: I am the main developer of HockeyKit and QuincyKit, and one of the developers of HockeyApp.
This was possible before TestFlight rolled out a service. The technique stemmed out of the enterprise distribution mechanism. Since 4.0 devices have supported install from web.
Remember - you still need to sign the beta distribution for a select set of UDIDs you can't just willy nilly install it on any device. All they are doing is taking the email the IPA step out of things.
See:
Update: I want to say that Test Flight is one of the most helpful tools I've used when developing though. Just taking the IPA emailing out of the picture was an understatement- I was just trying to call out the technical mechanism. They do a fantastic job managing the whole beta process. Getting new devices enrolled. Notifying users etc.
Testflight basically uses the normal Ad Hoc as already stated.
For this to work, you need the UDID for every device in order to add it to the Ad Hoc profile, re-compile the app with the new profile an redistribute the new build.
You can get the UDID with the help of the OTA Authentication Request. This is actually a step that is done in MDM before the actual profile is rolled out to the device. It basically asks the device for further information about itself and send it back to a self specified server.
The first step is documented here: Apple OTA Configuration
I guess Testflight uses this right after the registration process to collect the UDID, phone name, ...