Kubernetes Nginx Ingress Controller on NodePort

Done by disabling hostNetwork , and remove unnecessary privileges and capabilities:

C02W84XMHTD5:Downloads iahmad$ kubectl get deployments -n ingress-nginx -o yamlapiVersion: v1items:- apiVersion: extensions/v1beta1  kind: Deployment  metadata:    annotations:      deployment.kubernetes.io/revision: "1"    labels:      app.kubernetes.io/name: ingress-nginx      app.kubernetes.io/part-of: ingress-nginx    name: nginx-ingress-controller    namespace: ingress-nginx    resourceVersion: "68427"    selfLink: /apis/extensions/v1beta1/namespaces/ingress-nginx/deployments/nginx-ingress-controller    uid: 0b92b556-12fa-11ea-9d82-08002762a3c5  spec:    progressDeadlineSeconds: 600    replicas: 1    revisionHistoryLimit: 10    selector:      matchLabels:        app.kubernetes.io/name: ingress-nginx        app.kubernetes.io/part-of: ingress-nginx    strategy:      rollingUpdate:        maxSurge: 25%        maxUnavailable: 25%      type: RollingUpdate    template:      metadata:        annotations:          prometheus.io/port: "10254"          prometheus.io/scrape: "true"        creationTimestamp: null        labels:          app.kubernetes.io/name: ingress-nginx          app.kubernetes.io/part-of: ingress-nginx      spec:        containers:        - args:          - /nginx-ingress-controller          - --configmap=$(POD_NAMESPACE)/nginx-configuration          - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services          - --udp-services-configmap=$(POD_NAMESPACE)/udp-services          - --publish-service=$(POD_NAMESPACE)/ingress-nginx          - --annotations-prefix=nginx.ingress.kubernetes.io          env:          - name: POD_NAME            valueFrom:              fieldRef:                apiVersion: v1                fieldPath: metadata.name          - name: POD_NAMESPACE            valueFrom:              fieldRef:                apiVersion: v1                fieldPath: metadata.namespace          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1          imagePullPolicy: IfNotPresent          lifecycle:            preStop:              exec:                command:                - /wait-shutdown          livenessProbe:            failureThreshold: 3            httpGet:              path: /healthz              port: 10254              scheme: HTTP            initialDelaySeconds: 10            periodSeconds: 10            successThreshold: 1            timeoutSeconds: 10          name: nginx-ingress-controller          ports:          - containerPort: 80            name: http            protocol: TCP          - containerPort: 443            name: https            protocol: TCP          readinessProbe:            failureThreshold: 3            httpGet:              path: /healthz              port: 10254              scheme: HTTP            periodSeconds: 10            successThreshold: 1            timeoutSeconds: 10          resources: {}          securityContext:            runAsUser: 33          terminationMessagePath: /dev/termination-log          terminationMessagePolicy: File        dnsPolicy: ClusterFirst        restartPolicy: Always        schedulerName: default-scheduler        securityContext: {}        serviceAccount: nginx-ingress-serviceaccount        serviceAccountName: nginx-ingress-serviceaccount        terminationGracePeriodSeconds: 300  status:    availableReplicas: 1    conditions:    - lastTransitionTime: 2019-11-29T22:46:59Z      lastUpdateTime: 2019-11-29T22:46:59Z      message: Deployment has minimum availability.      reason: MinimumReplicasAvailable      status: "True"      type: Available    - lastTransitionTime: 2019-11-29T22:46:13Z      lastUpdateTime: 2019-11-29T22:46:59Z      message: ReplicaSet "nginx-ingress-controller-84758fb96c" has successfully progressed.      reason: NewReplicaSetAvailable      status: "True"      type: Progressing    observedGeneration: 1    readyReplicas: 1    replicas: 1    updatedReplicas: 1kind: Listmetadata:  resourceVersion: ""  selfLink: ""

and then creating a nodeport service pointing to the ingress controller ports:

C02W84XMHTD5:Downloads iahmad$ kubectl get svc -n ingress-nginx -o yamlapiVersion: v1items:- apiVersion: v1  kind: Service  metadata:    labels:      app.kubernetes.io/name: ingress-nginx      app.kubernetes.io/part-of: ingress-nginx    name: ingress-nginx    namespace: ingress-nginx    resourceVersion: "68063"    selfLink: /api/v1/namespaces/ingress-nginx/services/ingress-nginx    uid: 7aa425a4-12f9-11ea-9d82-08002762a3c5  spec:    clusterIP: 10.97.110.93    externalTrafficPolicy: Cluster    ports:    - name: http      nodePort: 30864      port: 80      protocol: TCP      targetPort: 80    - name: https      nodePort: 30716      port: 443      protocol: TCP      targetPort: 443    selector:      app.kubernetes.io/name: ingress-nginx      app.kubernetes.io/part-of: ingress-nginx    sessionAffinity: None    type: NodePort  status:    loadBalancer: {}kind: Listmetadata:  resourceVersion: ""  selfLink: ""C02W84XMHTD5:Downloads iahmad$ 

In the documentation about NodePort, you can find that this type can allocate ports from range 30000-32767.However there is a workaround. If you will add a special flag --service-node-port-range with requested range, admission controller allow you to create NodePort with Ports 80 and 443.

You will need to go to /etc/kubernetes/manifests/, edit kube-apiserver.yaml with sudo and add entry- --service-node-port-range=1-32767. After that you need to save it.

Now you will need to create service. To do that you need to edit this yaml and in ports add node port to spec.ports

Before:

 ports:    - name: http      port: 80      targetPort: 80      protocol: TCP    - name: https      port: 443      targetPort: 443      protocol: TCP

After:

  ports:  - name: http    nodePort: 80    port: 80    protocol: TCP    targetPort: 80  - name: https    nodePort: 443    port: 443    protocol: TCP    targetPort: 443

After thoses changes you can edit again kube-apiserver.yaml in /etc/kubernetes/manifests/ and comment it using # in the same line as - --service-node-port-range.

Then you will be able to curl this NodePort address and Node address.

EDIT:After clarification

Ingress can be deployed in two ways. The first one is deploy Nginx as Deamonset which is requiring hostPort inside configuration file. However there is another option, you can deploy Nginx as Deployment.

NodeIP and Known Port: Pods in the DaemonSet can use a hostPort, so that the pods are reachable via the node IPs. Clients know the list of node IPs somehow, and know the port by convention.

However in the bottom of the page you can find:

DaemonSets are similar to Deployments in that they both create Pods, and those Pods have processes which are not expected to terminate (e.g. web servers, storage servers).

Use a Deployment for stateless services, like frontends, where scaling up and down the number of replicas and rolling out updates are more important than controlling exactly which host the Pod runs on. Use a DaemonSet when it is important that a copy of a Pod always run on all or certain hosts, and when it needs to start before other Pods.

You need to deploy Ingress as Deployment and not as Deamonset.

Example of Nginx Deployment can be found here. As Deployment is not requiring hostPort you will be able to create pods without this parameter.