how do I enable mount propagation in Rancher - Kubernetes feature gates?

Hello and hope this helps someone, After much googling and help from awesome people at Rancher I got the solution for this.Here is what you can do to set the feature gates flags for the Kubernetes engine RKE.

step 1: Open Rancher2.0 UI

step 2: View cluster in API

step 3: Click edit and modify the rancherKubernetesEngineConfig input box

• Find the services key.

• Then add extra args for kubelet in below format

  "services": {"etcd": { "type": "/v3/schemas/etcdService" },"kubeApi": {    "podSecurityPolicy": false,    "type": "/v3/schemas/kubeAPIService",    "extraArgs": { "feature-gates": "PersistentLocalVolumes=true, VolumeScheduling=true,MountPropagation=true" }},"kubeController": { "type": "/v3/schemas/kubeControllerService" },"kubelet": {    "failSwapOn": false,    "type": "/v3/schemas/kubeletService",    "extraArgs": { "feature-gates": "PersistentLocalVolumes=true, VolumeScheduling=true,MountPropagation=true" }}

step 4: Click show request .. you get a curl command and json request.

step 5: Verify the request body data which will be shown.

step 6: Make sure the key's which are not applicable are set to null. e.g amazonElasticContainerServiceConfig, azureKubernetesServiceConfig, googleKubernetesEngineConfig all need to null for me.

step 7: Click send request

You should get a response with status code 201. And your cluster will start updating. You can verify that your cluster RKE has updated by viewing the Cluster in API again.

alternative (test is pending, blocked by https://github.com/rancher/rancher/issues/26261):

step 1: Open Rancher2.0 UI

step 2: Click edit for cluster (in menu with three vertical dots)

step 3: Click the button "edit as YAML" next to "Cluster Options"

step 4: find "services.kubelet" (or "services.WhatYouNeed")

step 5: add or edit extra_args similar to this:

    kubelet:      extra_args:        feature-gates: rancherKubernetesEngineConfig=true

(according to https://rancher.com/docs/rke/latest/en/config-options/services/services-extras/#extra-args)

step 6: click save (at bottom)

create new cluster

You can also edit "Cluster Options" as yaml when creating a new cluster. Probably this can also be included as an "rke template".

I was strugling with enabling feature gate TTLAfterFinished. (same as MountPropagation=true)

Rancher v2.3.3

step 1: Open Rancher2.3.3 UI

step 2: Click edit cluster

step 3: Click the button "edit as YAML" next to "Cluster Options"

step 4: You need to add feature gate to all cluster services (see below)

step 5: add or edit extra_args similar to this:

    ...        kube-api:          extra_args:            feature-gates: TTLAfterFinished=true    ...        kube-controller:          extra_args:            feature-gates: TTLAfterFinished=true    ...        kubelet:          extra_args:            feature-gates: TTLAfterFinished=true    ...  

Replace TTLAfterFinished with your feature gate. LIST HERE