How does NetTcpBinding(read WindowsStreamSecurityBindingElement) encrypt/sign messages?

The short answer is that, yes, with NTLM authentication the messages will still get signed and encrypted if you have set the Transport security ProtectionLevel to EncryptAndSign (the default).

Here's an outline of how it works:

• selecting Transport securityconfigures aWindowsStreamSecurityBindingElementin the channel stack. This inserts astream upgrade provider (see below)
• in the NetTcpBinding, messageexchange between the client andservice happens within the .NET MessageFraming Protocol, which provides bothmessage framing and a mechanism forclient and service to negotiatestream upgrades, the principal use ofwhich is to establish transportsecurity. If there is a streamupgrade provider configured in thechannel stack, this will be invokedduring the Preamble stage of theFraming Protocol when the clientopens the channel.
• the upgradeprovider forWindowsStreamSecurityBindingElement invokes an SSPI handshake between the client and the server using the SPNEGO security package: in the NetTcpBinding this will normally result in Kerberos being selected as the underlying security provider if available, but will choose NTLM if not.
• if NTLM is the resulting authentication provider, the SSPI handshake will involve the three-leg NTLM challenge-response exchange of tokens described in the NTLM specification. This protocol includes a mechanism for exchanging keys for message signing and encryption. Once the SSPI handshake has generated an appropriate security context, thereafter all messages exchanged are signed and encrypted in the sending channel stack's stream upgrade provider, and decrypted and verified in the receiving channel stack's stream upgrade provider, in each case by using calls to the NTLM security provider via the abstracted SSPI message support functions.

This is a Microsoft propriety implementation and not properly documented and perhaps on purpose to prevent intruders to take advantage of it.

As far as I know, this usually happens at the TCP level with a special token is generated by the user's credentials and passed along with the request. This is intercepted by windows security channel and authenticated against the AD.

This token is used as a key (or as a basis for generating the key) for encrypting the communication.

I think if you look at the TCP packet, you must be able to see the token - although I have never seen it.

If you are doing this all in code then you can find out the options here (search for 'NetTcpBinding'). Transport security is via Windows builtin TLS.

The diagram here should be helpful for your scenario.