calico network connectivity failing between pods and services and pods in different hosts

Calico configured in BGP mode requires all of the instances to be located in the same subnet to work out of the box.

To use calico with deployments that are split across multiple availability zones you must:

Disable AWS source / destination check (see here):

You can do that using AWS CLI:

    aws ec2 modify-instance-attribute --no-source-dest-check --instance-id          $EC2_INSTANCE_ID --region <REGION-WHERE-EC2-INSTANCE-IS-LAUNCHED>

Or using the AWS console:

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
2. In the navigation pane, choose Instances.
3. Select the NAT instance, choose Actions, Networking, Change Source/Dest. Check.
4. For the NAT instance, verify that this attribute is disabled. Otherwise, choose Yes, Disable.
5. If the NAT instance has a secondary network interface, choose it from Network interfaces on the Description tab and choosethe interface ID to go to the network interfaces page. ChooseActions, Change Source/Dest. Check, disable the setting, and choose Save.*

Enable IPIP encapsulation and outgoing NAT on your Calico IP pools

(IPPool) represents a collection of IP addresses from which Calicoexpects endpoint IPs to be assigned. (see here how to set it up)

, then all of the Kubernetes instances must be located in the same subnet for Calico to work out of the box.

To enable the “CrossSubnet” IPIP feature, configure your Calico IP pool resources to enable IPIP mode to “CrossSubnet” like in the example below:

apiVersion: projectcalico.org/v3kind: IPPoolmetadata:  name: ippool-multi-azspec:  cidr: 192.168.0.0/16  ipipMode: CrossSubnetEOF

Example above refers to AWS cloud configuration taken from the Calico documentation. Please note that Calico docs has also information about GCP, Azure and IBM.

Remark: If you face another problems going "the hard way" you may want to use as a reference another cluster created by following calico guides below:

• Getting started - openshift
• Getting started - self managed public cloud - GCE
• Getting started - self managed public cloud - AWS

Lastly, it is worth to check is also this very good document about calico routing modes (it shows also cross subnets ipip mode).