rails: LoadError when multiple threads try to load the same namespaced class

Version Information

Ruby: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17] Rails: Rails 5.2.0

Steps to reproduce

  • Create brand new rails project using rails new awesome_application
  • Create a directory called app/models/deep and a file called app/models/deep/beep.rb with the following beautifully succinct code:
module Deep
  Class Beep
  end
end
  • Open up the rails console and run Deep::Beep - all good!
  • Close and open up the rails console again and run 2.times { |i| Thread.new { Rails.application.executor.wrap { Deep::Beep } } } - oh no! Sometimes fails with:
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: already initialized constant Deep
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: previous definition of Deep was here
#<Thread:0x00007fd4795428b0@(irb):1 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	5: from (irb):1:in `block (2 levels) in irb_binding'
	4: from .../activesupport-5.2.0/lib/active_support/execution_wrapper.rb:87:in `wrap'
	3: from (irb):1:in `block (3 levels) in irb_binding'
	2: from .../activesupport-5.2.0/lib/active_support/dependencies.rb:193:in `const_missing'
	1: from .../bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/active_support.rb:43:in `load_missing_constant'
.../activesupport-5.2.0/lib/active_support/dependencies.rb:510:in `load_missing_constant': Unable to autoload constant Deep::Beep, expected .../app/models/deep/beep.rb to define it (LoadError)

Why does this happen?

There is a race condition where two threads will concurrently run the const_missing method for the Deep module, and both will run the autoload_module! method and create two separate Deep module objects on each thread - I believe it should be a single object for any given module.

Then, one of the threads will load the Beep class into its local module object using require_or_load (which is mutexed globally with Dependencies.load_interlock) and continue happily, while the other will run require_or_load, determine that Beep is already loaded, and then raise LoadError because its local module (from_mod in the source code) does not have the Beep class defined on it.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 17
  • Comments: 33 (3 by maintainers)

Most upvoted comments

An other workaround that works for us was to only use one thread in development to keep the eager_load = false.

# config/puma/development.rb
workers 1
threads 1, 1
  • Rails 5.0.7.2
  • Ruby 2.6.6
  • Puma 4.3.5

Ha! Yes, you are correct. That’s what I get for trying to be funny - my actual tests were using A and A::B, which is harder to mess up 😃

The issue still stands, so I updated the original post, thanks.

I’ve encountered the same issue on the project I’m working on. After some investigation I found the following. If we take this code as an example

# controllers/invoices_controller.rb
class InvoicesController < ApplicationController
  Billing::Invoice 
end

# models/billing/invoice.rb
module Billing
  class Invoice
  end
end

The process of loading Billing::Invoice constant can be described by following pseudocode:

  1. load const Billing (it is not resolved by ruby so it has to be autoloaded) 1.1. check if file billing.rb exists in autoload paths 1.2. check if directory billing exists in autoload paths (it does) 1.2.1. create module Billing
  2. load const Invoice 2.1. check if file billing/invoice.rb exists in autoload paths (it does) 2.1.1. load/require the file billing/invoice.rb 2.1.2. check if the constant Invoice is now defined on the Billing module and raise LoadError error if not

The issue with concurrent execution arises on step 1.2.1 as the code which creates “automatic” modules is not thread safe. See autoload_module! method

def autoload_module!(into, const_name, qualified_name, path_suffix)
  return nil unless base_path = autoloadable_module?(path_suffix)
  mod = Module.new
  into.const_set const_name, mod
  autoloaded_constants << qualified_name unless autoload_once_paths.include?(base_path)
  autoloaded_constants.uniq!
  mod
end

Rails does not check if the module is already defined and does not synchronise access to this method which in turn results in multiple instances of Billing module being created. On each of those instances the const_missing method is later invoked (in step 2) for the constant Invoice. The code that loads the Invoice constant is thread safe (see load_missing_constant and require_or_load methods) and loads the Invoice constant only once. The Invoice constant is then saved in the constants table of one of created Billing modules, but step 2.1.2 is invoked for each of them resulting in the LoadError error being raised.

It seems that the issue can be solved by simply making autoload_module! method thread safe. I’ve experimented with this solution and it appears to have resolved the issue.

I had to disable wrap_parameters as a workaround