sidekiq-unique-jobs: Duplicated jobs on the queue

Describe the bug

Multiple jobs with same parameters are added to the process queue.

Expected behavior No duplication of job with same argument are added to the queue.

Current behavior

We have multiple jobs with these options: sidekiq_options lock: :until_executed, unique_args: ->(args) { [args.first] }, on_conflict: :reject

ProcessObjectType1, ProcessObjectType2…

For some reason, one class: ProcessJourney has duplicated jobs in the queue (our queue should contain 200k items and it has 800k+, as jobs are duplicated).

What’s also strange is that they don’t seem to get processed at all, yet their number if going up and down (mostly growing). We have lots of dead jobs too, so some are getting rejected.

Worker class

# frozen_string_literal: true

module JourneyManagement
  module Job
    class ProcessJourney
      include Sidekiq::Worker
      # sidekiq_options queue: 'default'

      sidekiq_options lock: :until_executed, unique_args: ->(args) { [args.first] }, on_conflict: :reject

      # ------------------------------------------------------------------------------
      def perform(options)
        journey_id = options.fetch('journey_id')
        journey = Journey.find_by(id: journey_id)
        return false if journey.nil?

        puts "[SIDEKIQ][JourneyManagement][#{journey_id}][START]"

        start_time = Time.zone.now

        ret = JourneyManagement::ProcessJourneyService.new.call(journey: journey)

        puts "[SIDEKIQ][JourneyManagement][#{journey_id}][END] #{Time.zone.now - start_time}s"
      end
    end
  end
end

Additional context

[SIDEKIQ][JourneyManagement] number of messages in the log are incredible small. Like 10 processed in 5 minutes, where it should be thousands.

Worth noting that nothing has changed, the last release we made was last week and this suddenly started happening, this is blowing up our redis instance too as duplicates of this object are in the queue.

Gem:

gem 'sidekiq', '~> 6.1.2'
gem 'sidekiq-scheduler', '~> 2.2.2'
gem 'sidekiq-status', '~> 1.1.4'
gem 'sidekiq-unique-jobs', '~> 6.0.25'

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

Well, bottom line is that I was never able to bring stability to version 6 so I focused my efforts on a rewrite. These types of problems won’t happen with the next version.

That said, I get the resistance to upgrading. If you find out what went wrong, then let me know so that I can at least fix it for v7. Failing test scenarios are helpful here; even if it is just using the command line.

Version 6 unfortunately, is not stable. To give you a little more information, shopify is using unique jobs v7 beta so it can’t be all bad. 😉

Sorry about the trouble @jchatel

The whole reason for the v7 rewrite was to refactor the code in such a way that finding and fixing these types of errors wouldn’t be so hard. The v6 branch is the worst mistake I ever made as a software engineer. I thought I was clever at the time and it was released way ahead of time.

V7 takes this whole negative experience and I attempt to make something solid out of it. You should hopefully have way less of these problems in v7.

I understand if upgrading to a beta version is scary but everywhere I have tried I find it to be a ton more reliable. Any chance of getting you to help me roll out v7 by testing it and see if it solves the problem?

Admittedly, as has already been pointed out, the README could use some improvement. That said, it should be an automatic upgrade that requires very little from you apart from some configuration changes perhaps.

Let me know how you’d like to proceed.