rails: Rails 6 inconsistently overrides ActiveJob queue_adapter setting with TestAdapter

I had a similar problem with RSpec and request tests, I could workaround that with a little adaption of the code from @coorasse’s response above.

# spec/support/inline_active_job.rb

RSpec.configure do |config|
  config.around(:each, type: :request) do |example|
    original_queue_adapter = ActiveJob::Base.queue_adapter
    descendants = ActiveJob::Base.descendants + [ActiveJob::Base]

    ActiveJob::Base.queue_adapter = :inline
    descendants.each(&:disable_test_adapter)
    example.run
    descendants.each { |a| a.enable_test_adapter(ActiveJob::QueueAdapters::TestAdapter.new) }
    ActiveJob::Base.queue_adapter = original_queue_adapter
  end
end

This issue showed up for us because we use the :sidekiq adapter even in our test environment. We were testing Sidekiq queue length in two of our tests, one which was a request spec, and the other just a model spec. The model spec had the correct adapter, but the request spec did not. Just providing more context for anyone else who comes across this issue! Adding (ActiveJob::Base.descendants << ActiveJob::Base).each(&:disable_test_adapter) to config.before(:suite) in our spec_helper.rb fixed it.

We have the same issue with Rails 6.0.2.1. Workaround: adding in setup:

(ActiveJob::Base.descendants << ActiveJob::Base).each(&:disable_test_adapter)

(thanks @searls ❤️ )

This helped me:

config.before(:each) do
  (ActiveJob::Base.descendants << ActiveJob::Base).each(&:disable_test_adapter)
  ActiveJob::Base.queue_adapter = :sidekiq
end

@gmcgibbon Could we get this issue reopened?

I’d like to leave this here, since it could help somebody else. In our standard configuration we use rspec and we run system tests with an inline adapter, and other tests with test adapter. We had this RSpec configuration:

config.around(:each, :inline_jobs) do |example|
  ActiveJob::Base.queue_adapter = :inline
  example.run
  ActiveJob::Base.queue_adapter = :test
end

this needs to be changed to:

  config.around(:each, :inline_jobs) do |example|
    (ActiveJob::Base.descendants << ActiveJob::Base).each { |a| a.disable_test_adapter }
    ActiveJob::Base.queue_adapter = :inline
    example.run
    (ActiveJob::Base.descendants << ActiveJob::Base).each { |a| a.enable_test_adapter(ActiveJob::QueueAdapters::TestAdapter.new) }
    ActiveJob::Base.queue_adapter = :test
  end

Hi everyone! I’d love some feedback on this: https://github.com/rails/rails/pull/48585

By the way, for anyone looking to just disable this, one might write a setup hook like this one, invoking the disable_test_adapter method also added by the ActiveJob::TestHelper:

  def setup
    (ActiveJob::Base.descendants << ActiveJob::Base).each do |job_class|
      job_class.disable_test_adapter
    end
  ensure
    super
  end

The only thing that worked for me was something like:

def setup
  queue_adapter_changed_jobs.each(&:disable_test_adapter)
end

cause the line suggested in this thread

(ActiveJob::Base.descendants << ActiveJob::Base).each(&:disable_test_adapter)

broke other tests, that uses the test adapter and were executed after my test with the real queue adapter. I don’t really understand why it works that way, but for me it does.

OK, with my rspec setup, in an rspec/rails :system test, what’s happening is first the rspec before block runs.

THEN this Rails ActiveJob test_helper before_setup block runs: https://github.com/rails/rails/blob/master/activejob/lib/active_job/test_helper.rb#L36-L46

That before_setup calls enable_test_adapter with an argument being the method queue_adapter_for_test. But queue_adapter_for_test is hard-coded to ActiveJob::QueueAdapters::TestAdapter.new https://github.com/rails/rails/blob/master/activejob/lib/active_job/test_helper.rb#L63-L65

I am not sure if this is a bad interaction between rspec-rails and rails that needs to be fixed on rspec side… but I also don’t understand how this architecture allows you to change the test ActiveJob adapter at all, it seems to have a before_setup that is hard-coded to insist on the Test adapter (ie :test).

I don’t understand what the intent/purpose of that code in activejob/lib/active_job/test_helper.rb is, it’s very confusing.

The comments go on to say:

The adapter provided by this method must provide some additional methods from those expected of a standard <tt>ActiveJob::QueueAdapter</tt> in order to be used with the active job test helpers. Refer to <tt>ActiveJob::QueueAdapters::TestAdapter</tt>.

There is nothing talking about “additional methods” in the TestAdapter source/comments though that I can find.

I’m pretty bewildered. This is all very old code that predates Rails 6, so I’m not sure why things are breaking just in Rails 6, especially because this code doesn’t make any sense to me in the first place.

My workaround was to wrap the code inside a

perform_enqueued_jobs do
....
end

This way, any jobs that are enqueued inside the block, will be run automatically.

It is mysterious and frustrating!

It definitely matters what kind of test it was. So interstingly, there was a previous bug in Rails related to this that only applied to system tests, that was fixed in: https://github.com/rails/rails/pull/36283

However, that PR makes system tests “Inherit from ActiveSupport::TestCase instead of ActionDispatch::IntegrationTest” to fix the problem – which implies the problem might still be there in IntegrationTests, or perhaps Integration tests are intentionally not designed to allow queue_adapter setting?

@rept, you say you are using rspec requests tests… but guess what, rspec says: “Request specs provide a thin wrapper around Rails’ integration test”,

There are two things to tease apart here:

1. Separating rspec from rails – this is the rails issue tracker, if it’s an rspec-specific problem, or if it hasn’t been confirmed which it is, it’s especially less likely to result in anything here
2. What type of test are people having the problem in – this may be test-type specific

So, while a pain, if I were you and had time/energy and wanted to try to help get somewhere here, I’d try making a fresh empty Rails app that has nothing in it, and seeing if I can reproduce the problem without usingRspec, using a Rails integration test specifically – not rspec. (or if you’re having the problem in another test type, in that one).

Then report that with the reproduction (if you can do it as a self-contained test-case, even better) – either here, or make a new issue with more specific scope and concise problem report.

Sorry, I know a bug like this is annoying.

Rails still force TestAdapter to be used. None of the solutions above didn’t help, I can’t change the adapter nor before tests starts nor even just before perform_later call. Only spreading perform_enqueued_jobs in tests helped me.

TL; DR: Upgrading from Rails 5 to 6 caused this bug. However, I cannot confirm that 6-0-stable or master fixed the problem.

Background

When I enqueue a job via Ajax, I’m using the provider_job_id to set up a websocket listener to provide visual feedback on how this particular job is doing.

For some reasone, the provider_job_id is not properly kept between requests when I use the TestAdapter (not sure why, I think it’s simply not persisting it). So I’ve been using Sidekiq even in tests:

# config/application.rb
ActiveJob::Base.queue_adapter = :sidekiq

But I have Sidekiq run the jobs inline in feature tests (that way, the provider_job_id is reliably kept):

# spec/rails_helper.rb
RSpec.configure do |config|
  config.around sidekiq: true do |example|
    Sidekiq::Testing.inline! { example.run }
  end
end

So, my capybara tests started failing as of Rails 6:

RSpec.describe 'Documents', :js, :sidekiq, type: :system do
  it 'can Preview PDFs' do
    visit document_path create(:document)

    click_on 'PDF Preview'

    p ActiveJob::Base.queue_adapter  # <- ActiveJob::QueueAdapters::TestAdapter
    
    # This test passes if I uncommend the following line:
    # (ActiveJob::Base.descendants << ActiveJob::Base).each(&:disable_test_adapter)

    p ActiveJob::Base.queue_adapter  # <- Becomes Sidekiq only when disabling the test adapter

    click_on 'Download generated PDF'
  end
end

I could not even move the patch to the RSpec config.around block, it works only right here, in the test.

I started noticing in my test logs that, in Rails 6, the job is only enqueued but not run. (And when I wrapped the test in perform_enqueued_jobs do, the job ran, but the provider_job_id was not matching from when the job was enqueued compared to when the websocket channel is set up.)

Thank you @searls for the detailed bug report, can you confirm whether 6-0-stable fixed this for you?

@gmcgibbon Did you try the “Steps to reproduce” from the issue description on 6-0-master? I’ll try that.

I’m truly sorry I wasn’t there when the Rails 6 release candidates were battle tested 😢 Let me know if I can be of further assistance.

@benoittgt Happy to

We added a patch on RSpec to do that on system spec but maybe we could handle it for integration spec too.

See https://github.com/rspec/rspec-rails/pull/2242 and https://relishapp.com/rspec/rspec-rails/v/4-0/docs/system-specs/system-spec#the-activejob-queue-adapter-can-be-changed

Could you open an issue?

Thanks @beauraF. That’s precisely what #36283 was trying to sidestep for system test cases. I’m thinking we might have to change the behaviour of ActiveJob::TestHelper because it looks like its inclusion in integration tests is intentional. It seems we likely have to delay enabling the test adapter on jobs until just before we assert.

Backported in ea77dbffdc81d6e40bbde155961ab57f1450d1e4, 1a09b6ebfb75a26dbb4143bf220733aefa854dd8, and 109aad7ca93f0bcb2319df19b45f05b6794b3b92. Let me know if you’re still having issues against 6-0-stable and we can reopen. Thanks!