Can flink run mutliple same jobs to achieve pseudo dynamic scaling? Can flink run mutliple same jobs to achieve pseudo dynamic scaling? kubernetes kubernetes

Can flink run mutliple same jobs to achieve pseudo dynamic scaling?


Is there a reason why you don't want to modify parallelism by stopping the job?

You could do this however you would effectively be splitting your data across the various jobs. So not only would you be incurring the cost of now needing to understand your throughput across multiple jobs to efficiently autoscale but you would make it such that any stateful processing that is done would result in incorrect/inconsistent results.