iOS app SSL handshake failed

On iOS >= 10.3, after installing the Charles Proxy certificate (as per @ashish-verma's excellent answer), you also need to enable it through Settings -> General -> About -> Certificate Trust Settings:

Full setup here: http://www.devsbedevin.net/debugging-ios-ssl-traffic/

If installing the certificate still doesn't help you, then it may be due to SSL Pinning.

From charlesproxy.com:

SSL Pinning Note that some apps implement SSL certificate pinning which means they specifically validate the root certificate. Because the app is itself verifying the root certificate it will not accept Charles's certificate and will fail the connection. If you have successfully installed the Charles root SSL certificate and can browse SSL websites using SSL Proxying in Safari, but an app fails, then SSL Pinning is probably the issue.

Perhaps testing on an actual device will give more insight?

To do that in Charles:

1. Goto Proxy: Proxy Settings...
2. Proxies tab: Enter "8888" in the HTTP Proxy port: field
3. SSL Tab: make sure the "Enable SSL Proxying" is on

On your dev machine:

1. Goto System Preferences: Network: Wifi: Advanced: TCP/IP Tab: Make note of the IPV4 Address: (i.e. 10.0.1.101)

On your iDevice:

1. Goto http://charlesproxy.com/charles.crt from your device and download Charles SSL certificate
2. Okay the certificate as trusted (be sure to disable or remove it once you're done)
3. Goto the Settings App: Wifi: Select the detailed disclosure button (round blue button with the white arrow) for your network
4. Under HTTP Proxy, select "Manual"
5. Enter in the address from step (dev machine 1.) into the Server field, and the port number from step (charles 2.)

Finally:

1. Restart Charles
2. Run Safari on your iDevice to test. This should prompt Charles to display a dialog box to allow or deny SSL proxying. Obviously allow and you should be good to go.

More Info at Charles's site: http://www.charlesproxy.com/documentation/proxying/ssl-proxying/

Also, are you getting a response to the connection:(NSURLConnection*)connection didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge*)challenge method?