EKS - How to annotate some nodes in USERDATA?

You can add node labels, taints, etc by using the --kubelet-extra-args option on the bootstrap.sh invokation as you guessed. For an example, see the AWS Blog post: Improvements for Amazon EKS Worker Node Provisioning

Use a USERDATA script similar to the following:

      UserData: !Base64        "Fn::Sub": |          #!/bin/bash          set -o xtrace          /etc/eks/bootstrap.sh ${ClusterName} ${BootstrapArguments}          /opt/aws/bin/cfn-signal --exit-code $? \                   --stack  ${AWS::StackName} \                   --resource NodeGroup  \                   --region ${AWS::Region}

The above is a fragment from the CloudFormation template. Of course you can make your script more complex, with security hardening, etc. if you so desire.

For a complete CloudFormation template, download the sample from AWS:

curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2019-11-15/amazon-eks-nodegroup.yaml

No, it not possible.

The list of supported parameters for the bootstrap script:

--use-max-pods Sets --max-pods for the kubelet when true. (default: true)--b64-cluster-ca The base64 encoded cluster CA content. Only valid when used with --apiserver-endpoint. Bypasses calling \"aws eks describe-cluster\"--apiserver-endpoint The EKS cluster API Server endpoint. Only valid when used with --b64-cluster-ca. Bypasses calling \"aws eks describe-cluster\"--kubelet-extra-args Extra arguments to add to the kubelet. Useful for adding labels or taints.--enable-docker-bridge Restores the docker default bridge network. (default: false)--aws-api-retry-attempts Number of retry attempts for AWS API call (DescribeCluster) (default: 3)--docker-config-json The contents of the /etc/docker/daemon.json file. Useful if you want a custom config differing from the default one in the AMI

It is absolutely possible. Here is part of my example userdata, specifically useful if you want to run both OnDemand and Spot instance. In my example I am adding lifecycle node label which changes based on the type. See below:

--use-max-pods 'true' \--kubelet-extra-args ' --node-labels=lifecycle=OnDemand \--system-reserved cpu=250m,memory=0.2Gi,ephemeral-storage=1Gi \--kube-reserved cpu=250m,memory=1Gi,ephemeral-storage=1Gi \--eviction-hard memory.available<0.2Gi,nodefs.available<10% \--event-qps 0'

I hope that gives you a nice example.