How can I use autoscale from K8s in a Redis Cluster if I'm in a Spring boot application with Spring Data (Jedis) connect in a Redis Cluster?

Good question 💯.

The short answer is that you typically don't do autoscaling with stateful apps like Redis since you have to be careful about not corrupting your data. Most of the time you migrate and shard your data, i.e multiple clusters, with different segments of your data, etc.

Having said that, there is no silver bullet redis autoscaling solution but it's doable with a lot of monitoring and testing 🦄. A challenge here is that sentinels change the master in case of failover so your solution needs to be able to determine or monitor who the master is at a certain interval, this is very critical during downscales. Redis has written a pretty good guide on how to create clients which you will probably have to do/understand if you want a reliable autoscaling solution.

So the idea here 💡 is that you start with a set of sentinel/redis nodes managed by a Kubernetes Operator. With some config like this:

apiVersion: databases.spotahome.com/v1kind: RedisFailovermetadata:  name: redisfailoverspec:  sentinel:    replicas: 3    resources:      requests:        cpu: 100m      limits:        memory: 100Mi  redis:    replicas: 3    resources:      requests:        cpu: 100m        memory: 100Mi      limits:        cpu: 400m        memory: 500Mi

Then maybe modify the controller of this operator to do autoscale based on certain metrics (CPUs, Memory, Storage, etc).

The moment there is an autoscale operation you will have to do a configuration change in your Spring boot application to account for this change (say the ConfigMap of your application). For example, automatically change the value of this:

spring:  cache:    type: redis  redis:    port: 6666    password: 123pwd    sentinel:      master: masterredis      nodes:        - 10.0.0.16        - 10.0.0.17        - 10.0.0.18    lettuce:      shutdown-timeout: 200ms

Now after the config change, you need to do a rolling restart to prevent any downtime. The best thing, in my opinion, to do this in Kubernetes is just to have another Operator (or extend the Redis operator) for your application, that has a controller, to automatically detect when there is scaling operation, does the ConfigMap change, and finally does the rolling restart of your app. Your scaling operations need to allow enough time for balancing of data and also the rolling restart to prevent any thrashing/starvation and possible downtime/data corruption.

✌️☮️