rails: Scheduled jobs set by resque-scheduler are not working with ActiveJob in Rails 4.2

I am using resque, and I am attempting to use resque-scheduler to schedule jobs. I have a schedule that get loaded and the scheduler runs, and even looks like it is running the jobs but it doesn’t do anything.

resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Starting
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Loading Schedule
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Scheduling friends 
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Schedules Loaded
resque-scheduler: [INFO] 2014-09-16T01:54:55-07:00: queueing FriendsJob (friends)

I can enqueue jobs like this and they get processed.

TestJob.enqueue(params[:id])

and I get this output in the worker log

got: (Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])
** [01:24:01 2014-09-16] 54841: resque-1.25.2: Processing default since 1410855841  [ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper]
** [01:24:01 2014-09-16] 54841: Running before_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]
** [01:24:01 2014-09-16] 54841: resque-1.25.2: Forked 54882 at 1410855841
** [01:24:01 2014-09-16] 54882: Running after_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]
Hello World!!
** [01:24:01 2014-09-16] 54882: done: (Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])

But when I try to schedule a job, it looks like the are getting enqueue but they well not doing anything.

Here is the schedule.yml

friends:
  every: "30s"
  queue: "friends"
  class: "FriendsJob"
  args: 8
  description: "Friends jobs scheduler"

Here is the output from the scheduled job.

** [01:23:36 2014-09-16] 54841: got: (Job{friends} | FriendsJob | [8])
** [01:23:36 2014-09-16] 54841: resque-1.25.2: Processing friends since 1410855816 [FriendsJob]
** [01:23:36 2014-09-16] 54841: Running before_fork hooks with [(Job{friends} | FriendsJob | [8])]
** [01:23:36 2014-09-16] 54841: resque-1.25.2: Forked 54880 at 1410855816
** [01:23:36 2014-09-16] 54880: Running after_fork hooks with [(Job{friends} | FriendsJob | [8])]
** [01:23:36 2014-09-16] 54880: done: (Job{friends} | FriendsJob | [8])

After reading this http://dev.mikamai.com/post/96343027199/rails-4-2-new-gems-active-job-and-global-id I am suspecting it has something to do with ActiveJob and GlobalId.

Take a look at the difference enqueued

** [01:24:01 2014-09-16] 54841: Running before_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]

vs scheduled

** [01:23:36 2014-09-16] 54841: Running before_fork hooks with [(Job{friends} | FriendsJob | [8])]

The jobs themselves were generated via

 rails g job <JobName>

They look like this:

class TestJob < ActiveJob::Base
  queue_as :default
  def perform(friend_id)
    friend = Friend.find(friend_id)
    name = friend.name.swapcase
    puts "Hello World!!"
  end
end


class FriendsJob < ActiveJob::Base
  queue_as :friends
  def perform(friend_id)
    friend = Friend.find(friend_id)
    name = friend.name.swapcase
    puts "Hello World!!"
  end
end

After, writing most of this, I’ve have removed 'require “active_job/railtie” ’ and rewritten the jobs in the resque way and now they work as expected. So I think that the problem is with active_job and the way that scheduled jobs get queued.

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Reactions: 5
  • Comments: 16 (7 by maintainers)

Commits related to this issue

Most upvoted comments

@ryanwjackson - I liked your idea so much I’m turning it into a gem: https://github.com/JustinAiken/active_scheduler

+7
JustinAiken on Mar 10, 2015

I’ve managed to force scheduler to schedule jobs through ActiveJob using extension support approach. Answer on StackOverflow.

+1
svetam on Jan 31, 2018

Using rails 5.1.3 Using resque 1.27.4 Using resque-scheduler 4.3.0

lib/tasks/rescue.rake

require 'resque/scheduler/tasks'
require 'resque/tasks'
namespace :resque do
  desc 'resque'
  task setup: :environment do
    require 'resque'
    require 'resque-scheduler'
    ENV['QUEUE'] ||= '*'
  end
end

config/redis.yml

default: &default
  host: 127.0.0.1
  port: 6379
development:
  <<: *default
  db: 0
test:
  <<: *default
  db: 1
production:
  <<: *default
  db: 2
  host: <%= ENV['REDIS_SERVICE_HOST'] %>
  port: <%= ENV['REDIS_SERVICE_PORT'] %>

config/initializers/resque.rb

REDIS_CONFIG = YAML.load(ERB.new(File.read("#{Rails.root}/config/redis.yml")).result)[Rails.env]
Resque.redis = Redis.new(REDIS_CONFIG)

# dynamically change the schedule
Resque::Scheduler.dynamic = true

# resque-scheduler needs to know about jobs unless +queue+ is set
require "#{Rails.root}/app/jobs/execute_active_job.rb"

Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }

app/jobs/execute_active_job.rb

module ExecuteActiveJob
  @queue = :execute_active_job
  def self.perform(klass, *args)
    klass = Object.const_get(klass)
    args.empty? ? klass.perform_later() : klass.perform_later(*args)
  end
end

app/jobs/test_job.rb

class IFailed < StandardError; end
class TestJob < ApplicationJob
  queue_as :default

  def perform(*args)
    args.empty? ? raise(IFailed, 'i failed with no args') : raise(IFailed, args[0])
  end
end

config/resque_schedule.yml

# TestJob will raise error with first argument
do_test_job_with_args:
  every: 10s
  class: ExecuteActiveJob
  args:
    - TestJob
    - "I am king"
  description: Kicks off test with argument
do_test_job_without_args:
  every: 10s
  class: ExecuteActiveJob
  args: TestJob
  description: Kicks off test without argument
+1
jeremiahgithub on Oct 13, 2017

After much frustration, I ended up going with the wrapper suggested by @JustinAiken. I wrote the following:

class JobWrapper
  def self.perform(args)
    Object.const_get(args['job_class']).perform_later
  end

  def self.wrap(schedule)
    # from: https://github.com/rails/rails/issues/16933#issuecomment-58945932
    schedule = HashWithIndifferentAccess.new(schedule)
    schedule.each do |k, v|
      next unless v[:class] != 'JobWrapper'
      q = v[:queue] || 'default'

      schedule[k] = {
        class: 'JobWrapper',
        description: v[:description],
        queue: q,
        cron: v[:cron],
        args: [{
          job_class: v[:class],
          queue_name: q,
          arguments: v[:arguments]
        }]
      }
    end
  end
end

Which allows you to keep the regular Resque schedule format and do the following in your setup_schedule Rake task:

Resque.schedule = JobWrapper.wrap(YAML.load_file("#{Rails.root}/config/resque_schedule.yml") || {})

Seems to work for me, but lmk if I’m missing something.

+1
ryanwjackson on Mar 10, 2015

As a workaround, I’m queing Resque schedules (with no args) to an ActiveJob like this:

check_goals:
  queue: "goals"
  every: "5m"
  class: "ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper"
  args:
    -
      job_class: CheckGoalJob
      job_id: #{SecureRandom.uuid}
      queue_name: goals
      arguments:
        -
  description: "Checks all checkable goals to see if they're achieved"
+1
JustinAiken on Oct 13, 2014
Read more comments on GitHub
← rails: save (ActiveRecord::Persistence) can return true even when the model is not saved, and seems to break the principle of "least surprise"
rails: scopes combine conditions on same attribute incorrectly  →