sidekiq-cron: Sidekiq::Cron.configure cron_schedule_file does not work as described in the readme for version 2.0.0.rc2
(I’m going to post a bit of extraneous info because it might help someone else that is confused a bit by the readme and might make the same assumptions I did. I’m also not sure if this issue applies to any of the other config variables as well, but defaults work for me so I haven’t tested beyond cron_schedule_file yet)
I initially tried setting up my config/initializers/sidekiq-cron.rb file like so:
Sidekiq::Cron.configure do |config|
if Rails.env.production?
Rails.logger.info "\n========= Sidekiq::Cron - loading production schedule"
config.cron_poll_interval = 60 # Default is 30
config.cron_schedule_file = 'config/schedules/jobs_production.yml'
elsif Rails.env.staging?
Rails.logger.info "\n========= Sidekiq::Cron - loading staging schedule"
config.cron_schedule_file = 'config/schedules/jobs_staging.yml'
elsif Rails.env.development?
Rails.logger.info "\n========= Sidekiq::Cron - loading development schedule"
config.cron_schedule_file = 'config/schedules/jobs_development.yml'
end
end
But this would never load the appropriate cron jobs defined in the environment specific yml files. No errors or other messages indicating failure.
On a whim I renamed one of the yml files to the default “config/schedule.yml” and restarted sidekiq, and it worked (created the defined cron job and began executing it at defined interval).
Then I thought mabye it was the subdirectory, so I tried using:
config.cron_schedule_file = 'config/jobs_schedule_development.yml'
And this does NOT work either.
So with 2.0.0.rc2 (which if you want to follow the readme, you have to use instead of 1.12 to do the the Sidekiq::Cron.configure initializer at all, per this issue ) the only schedule file that works is the default config/schedule.yml.
Side note/gotcha: cron jobs scheduled with config/schedule.yml (or maybe with any use of Sidekiq::Cron ???) do not get deleted when sidekiq restarts, even if you delete the schedule file. You have to explicitly destroy the job somewhere in your code.
Note also that Sidekiq::Cron::Job.destroy_all! destroys ALL jobs, not just those scheduled by sidekiq-cron, so if you have a pending perform_later on a non-cron job somewhere before you call that (e.g. in other startup config/intiializers), you will destroy that explicit job perform_later call too.
Ultimately the solution that worked for me was to piggyback in my config/initializers/sidekiq.rb and manually load the schedule yml file I needed, like so:
Sidekiq.configure_server do |config|
if Rails.env.production?
schedule_file = "config/schedules/jobs_production.yml"
elsif Rails.env.staging?
schedule_file = "config/schedules/jobs_staging.yml"
elsif Rails.env.development?
schedule_file = "config/schedules/jobs_development.yml"
else
schedule_file = "config/schedules/jobs_test.yml"
end
config.on(:startup) do
# Destroy any previously scheduled jobs
Sidekiq::Cron::Job.destroy_all! # WARNING: This apparently destroys ALL pending jobs, not just those scheduled by sidekiq-cron!!
if File.exist?(schedule_file)
Rails.logger.info "\n========= Sidekiq - loading jobs schedule from #{schedule_file}"
schedule = YAML.load_file(schedule_file)
if schedule.present?
Sidekiq::Cron::Job.load_from_hash!(schedule, source: "schedule")
else
Rails.logger.warn "\n========= Sidekiq - No job definitions found in #{schedule_file}! Skipping cron scheduling."
end
end
# Any other non-cron startup jobs you need
OtherStartupJob.perform_later
end
Using this setup, it seems to reliably handle restarts in a way you’d expect if you make changes to the scheduling files.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (14 by maintainers)
Hi @tttffff,
Thank you for sharing your insights and for being thoughtful about checking in - I really appreciate it! It does seem like a cleaner and more robust approach to ensure the schedule loads correctly after the initializers.
Thanks again for stepping in with such a well-considered solution!
Hi @satyakampandya I have recently come across this issue but I do not want to step on your toes.
I think the best solution would be to run all the loading code inside the sidekiq startup callback. As that code is only ran after the initializers, we will have the correct file name set at the time it is ran. There would then be no issue with trying to load the default schedule only to then run the custom schedule. I also think it is a cleaner approach.
It only means moving the logic inside the
config.on(:startup)block, and tests should be simple to show that it works.I was planning on raising a PR for this after #549 is reviewed, but wanted to check with you. Do you think the above approach is good? Do you still want to take this, or is it okay if I do?
Let me know if you have any question on my approach.
Hi @markets Yes, I am interested to fix this. I would start afresh this week.
Hi @markets,
Apologies for the delay - I’ve been tied up with some other tasks and haven’t had the chance to revisit it yet. As far as I recall, there were some failing unit tests in my branch that I still need to address. I’ll be busy next week, but I’ll make sure to look into it after that and update you accordingly.
@markets Sure, I’ll check it.
Update
@markets Please review this branch. I’m implementing schedule loading immediately upon assignment. The default schedule file loads first (if it exists), and if the user changes it in the Sidekiq::Cron.configure block, the custom schedule file will load instead, enhancing flexibility.
If a custom schedule file is used, ensure that config/schedule.yml does not exist. If both are present, the default file will load first, and existing schedules will be deleted before the custom schedule is applied.