kubernetes securitycontext runAsNonRoot Not working

Nginx service will expect a read and write permission to its configuration path (/etc/nginx) by default non root user would have that access to the path that is the reason it is failing.You just set runAsNonRoot but you can't expect or guarantee that container will start the service as user 1001. Please try setting runAsUser explicitly to 1001 like below, this should resolve your issue.

apiVersion: v1kind: Podmetadata:  name: buggypodspec:  containers:    - name: container      image: nginx      securityContext:                runAsUser: 1001 

I try to run the pod based on your requirement. And the reason it failed is the Nginx require to modify some configuration in /etc/ owned by root and when you runAsNonRoot it fails as it cannot edit the Nginx default config.

This is the error you actually get when you run it.

10-listen-on-ipv6-by-default.sh: error: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh/docker-entrypoint.sh: Configuration complete; ready for start up2020/08/13 17:28:55 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:22020/08/13 17:28:55 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

The spec I ran.

apiVersion: v1kind: Podmetadata:  creationTimestamp: null  labels:    run: buggypod  name: buggypodspec:  securityContext:    runAsNonRoot: true    runAsUser: 1000  containers:  - image: nginx    name: buggypod    resources: {}  dnsPolicy: ClusterFirst  restartPolicy: Alwaysstatus: {}

My suggestion is you create a custom Nginx image with a Dockerfile that also creates user and provides permissions to the folders /var/cache/nginx, /etc/nginx/conf.d, /var/log/nginx for the newly created user. Such that you achieve running the container as Non-Root.