Is there a way to prevent envoy from adding specific headers?

As I mentioned in comments there is related github issue about that.

Is there a way to prevent envoy from adding specific headers?

There is istio dev @howardjohn comment about that

We currently have two options:

• EnvoyFilter
• Alpha api

There will not be a third; instead we will promote the alpha API.

So the first option would be envoy filter.

There are 2 answers with that in above github issue.

Answer provided by @jh-sz

In general, use_remote_address should be set to true when Envoy is deployed as an edge node (aka a front proxy), whereas it may need to be set to false when Envoy is used as an internal service node in a mesh deployment.

apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:  name: xff-trust-hops  namespace: istio-systemspec:  workloadSelector:    labels:      istio: ingressgateway  configPatches:  - applyTo: NETWORK_FILTER    match:      context: ANY      listener:        filterChain:          filter:            name: "envoy.http_connection_manager"    patch:      operation: MERGE      value:        typed_config:          "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager"          use_remote_address: true          xff_num_trusted_hops: 1

AND

Answer provided by @vadimi

apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:  name: my-app-filterspec:  workloadLabels:    app: my-app  filters:  - listenerMatch:      portNumber: 5120      listenerType: SIDECAR_INBOUND    filterName: envoy.lua    filterType: HTTP    filterConfig:      inlineCode: |        function envoy_on_request(request_handle)          request_handle:headers():replace("x-forwarded-proto", "https")        end        function envoy_on_response(response_handle)        end

The second option would be Alpha api, this feature is actively in development and is considered pre-alpha.

Istio provides the ability to manage settings like X-Forwarded-For (XFF) and X-Forwarded-Client-Cert (XFCC), which are dependent on how the gateway workloads are deployed. This is currently an in-development feature. For more information on X-Forwarded-For, see the IETF’s RFC.

You might choose to deploy Istio ingress gateways in various network topologies (e.g. behind Cloud Load Balancers, a self-managed Load Balancer or directly expose the Istio ingress gateway to the Internet). As such, these topologies require different ingress gateway configurations for transporting correct client attributes like IP addresses and certificates to the workloads running in the cluster.

Configuration of XFF and XFCC headers is managed via MeshConfig during Istio installation or by adding a pod annotation. Note that the Meshconfig configuration is a global setting for all gateway workloads, while pod annotations override the global setting on a per-workload basis.

The reason this happens is most likely because you have one or more proxies in front of Envoy/Istio.

You need to tell Envoy how many proxies you have in front of it so that it can set forwarded headers correctly (such as X-Forwarded-Proto and X-Forwarded-For).

In Istio 1.4+ you can achieve this with an Envoy filter:

apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:  name: xff-trust-hops  namespace: istio-systemspec:  workloadSelector:    labels:      istio: ingressgateway  configPatches:  - applyTo: NETWORK_FILTER    match:      context: ANY      listener:        filterChain:          filter:            name: "envoy.http_connection_manager"    patch:      operation: MERGE      value:        typed_config:          "@type": "type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager"          use_remote_address: true          xff_num_trusted_hops: 1 # Change as needed

Note that if you have multiple proxies in front of Envoy you have to change the xff_num_trusted_hops variable to the correct amount. For example if you have a GCP or AWS cloud load balancer, you might have to increase this value to 2.

In Istio 1.8+, you will be able to configure this via the Istio operator instead, example:

apiVersion: install.istio.io/v1alpha1kind: IstioOperatorspec:  meshConfig:    defaultConfig:      gatewayTopology:        numTrustedProxies: 1 # Change as needed

More information is available here.