NEPacketTunnelProvider Sniffer iOS

I've published a Beta Proxyman iOS (website) - a Network Sniffer by using NEPacketTunnelProvider and Network Extension, so I might have experienced to answer some of your questions.

IP Package, IP Diagram, DNS, How to parse it?

Luckily, there is another way to set up a NEPacketTunnelProvider to provide you with an HTTP Message, not IP Package (it's too low-level, and you have to deal with the Parser, DNS, ...)

HTTP Message is easier to parse because there are plenty of reliable libraries (e.g. http-parser from nodeJS)

1. How to build a Network Sniffer on iOS?

It's a complicated question to answer, I would break it into small chunks:

MitM / Proxy Server

Firstly, you need a working MitM Proxy Server, which is capable of proxying and intercepting the HTTP/HTTPS Traffic. You can implement it by using SwiftNIO or CocoaAsyncSocket.

How does it work?

In general, the data flow might look like this:

The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server (in the Network Extension) -> Mitm/Proxy Server starts intercepting or monitoring the traffic -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.

From the main app, you can receive the data by reading the local database.

The reason why we need a local database is that the Network Extension and the Main app are two different processes, so they could not communicate directly like a normal app.

Show me the code?

In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:

    private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {    let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")    /* proxy settings */    let proxySettings: NEProxySettings = NEProxySettings()    proxySettings.httpServer = NEProxyServer(        address: proxyHost,        port: proxyPort    )    proxySettings.httpsServer = NEProxyServer(        address: proxyHost,        port: proxyPort    )    proxySettings.autoProxyConfigurationEnabled = false    proxySettings.httpEnabled = true    proxySettings.httpsEnabled = true    proxySettings.excludeSimpleHostnames = true    proxySettings.exceptionList = [        "192.168.0.0/16",        "10.0.0.0/8",        "172.16.0.0/12",        "127.0.0.1",        "localhost",        "*.local"    ]    settings.proxySettings = proxySettings    /* ipv4 settings */    let ipv4Settings: NEIPv4Settings = NEIPv4Settings(        addresses: [settings.tunnelRemoteAddress],        subnetMasks: ["255.255.255.255"]    )    ipv4Settings.includedRoutes = [NEIPv4Route.default()]    ipv4Settings.excludedRoutes = [        NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),        NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),        NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")    ]    settings.ipv4Settings = ipv4Settings    /* MTU */    settings.mtu = 1500    return settings}

Then start a VPN,

let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)// StartsetTunnelNetworkSettings(networkSettings) { // Handle success }

Then forward the package to your local proxy server:

let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)    packetFlow.readPackets {[weak self] (packets, protocols) in        guard let strongSelf = self else { return }        for packet in packets {            strongSelf.connection.write(packet, completionHandler: { (error) in            })        }        // Repeat        strongSelf.readPackets()    }

From that, your local server can receive the packages then forwarding to the destination server.

Don't forget to save all traffic log to the local database, then notifying the main app to reload it.

One last question: Charles proxy is in most cases able to show the hostname of the target. I'm currently just able to see destination ip addresses (which aren't real destination ip addresses, but the address of my DNS server). How am I able to see the hostname as human readable text? Does Charles do a nslookup somehow? Does Charles obtain that information out of the datagrams?

Since we don't deal with IP Package, we don't need to implement the DNS Resolver. If you need a DNS, you can config like the following code:

        let dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "1.1.1.1"])    settings.dnsSettings = dnsSettings

As we receive the HTTP Message package, you can get hostname for free (From the Request's URL or Host Header)

Hope that my answer could help you.