How to constrain delayed_job processing based on kubernetes cluster How to constrain delayed_job processing based on kubernetes cluster kubernetes kubernetes

How to constrain delayed_job processing based on kubernetes cluster


Eventually, I ended up with the following solution. Add a varchar column named cluster to the delayed_jobs table and BOOM. Works like a charm.

require 'delayed/backend/active_record'module Delayed  module Backend    module ActiveRecord      class Configuration        attr_accessor :cluster      end      # A job object that is persisted to the database.      # Contains the work object as a YAML field.      class Job < ::ActiveRecord::Base        READY_SQL = <<~SQL.squish.freeze          ((cluster = ? AND run_at <= ? AND (locked_at IS NULL OR locked_at < ?)) OR locked_by = ?) AND failed_at IS NULL        SQL        before_save :set_cluster        def self.ready_to_run(worker_name, max_run_time)          where(READY_SQL, cluster, db_time_now, db_time_now - max_run_time, worker_name)        end        # When a worker is exiting, make sure we don't have any locked jobs.        def self.clear_locks!(worker_name)          where(cluster: cluster, locked_by: worker_name)            .update_all(locked_by: nil, locked_at: nil) # rubocop:disable Rails/SkipsModelValidations        end        def self.cluster          Delayed::Backend::ActiveRecord.configuration.cluster        end        def set_cluster          self.cluster ||= self.class.cluster        end      end    end  endendDelayed::Backend::ActiveRecord.configuration.cluster = ENV['CLUSTER'] if ENV['CLUSTER']