rails: LoadError when multiple threads try to load the same namespaced class
Version Information
Ruby: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17]
Rails: Rails 5.2.0
Steps to reproduce
- Create brand new rails project using
rails new awesome_application
- Create a directory called
app/models/deep
and a file calledapp/models/deep/beep.rb
with the following beautifully succinct code:
module Deep
Class Beep
end
end
- Open up the rails console and run
Deep::Beep
- all good! - Close and open up the rails console again and run
2.times { |i| Thread.new { Rails.application.executor.wrap { Deep::Beep } } }
- oh no! Sometimes fails with:
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: already initialized constant Deep
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: previous definition of Deep was here
#<Thread:0x00007fd4795428b0@(irb):1 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
5: from (irb):1:in `block (2 levels) in irb_binding'
4: from .../activesupport-5.2.0/lib/active_support/execution_wrapper.rb:87:in `wrap'
3: from (irb):1:in `block (3 levels) in irb_binding'
2: from .../activesupport-5.2.0/lib/active_support/dependencies.rb:193:in `const_missing'
1: from .../bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/active_support.rb:43:in `load_missing_constant'
.../activesupport-5.2.0/lib/active_support/dependencies.rb:510:in `load_missing_constant': Unable to autoload constant Deep::Beep, expected .../app/models/deep/beep.rb to define it (LoadError)
Why does this happen?
There is a race condition where two threads will concurrently run the const_missing
method for the Deep
module, and both will run the autoload_module!
method and create two separate Deep
module objects on each thread - I believe it should be a single object for any given module.
Then, one of the threads will load the Beep
class into its local module object using require_or_load
(which is mutexed globally with Dependencies.load_interlock
) and continue happily, while the other will run require_or_load
, determine that Beep
is already loaded, and then raise LoadError
because its local module (from_mod
in the source code) does not have the Beep
class defined on it.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 17
- Comments: 33 (3 by maintainers)
An other workaround that works for us was to only use one thread in development to keep the
eager_load = false
.Ha! Yes, you are correct. That’s what I get for trying to be funny - my actual tests were using
A
andA::B
, which is harder to mess up 😃The issue still stands, so I updated the original post, thanks.
I’ve encountered the same issue on the project I’m working on. After some investigation I found the following. If we take this code as an example
The process of loading
Billing::Invoice
constant can be described by following pseudocode:Billing
(it is not resolved by ruby so it has to be autoloaded) 1.1. check if filebilling.rb
exists in autoload paths 1.2. check if directorybilling
exists in autoload paths (it does) 1.2.1. create moduleBilling
Invoice
2.1. check if filebilling/invoice.rb
exists in autoload paths (it does) 2.1.1. load/require the filebilling/invoice.rb
2.1.2. check if the constantInvoice
is now defined on theBilling
module and raiseLoadError
error if notThe issue with concurrent execution arises on step 1.2.1 as the code which creates “automatic” modules is not thread safe. See
autoload_module!
methodRails does not check if the module is already defined and does not synchronise access to this method which in turn results in multiple instances of
Billing
module being created. On each of those instances theconst_missing
method is later invoked (in step 2) for the constantInvoice
. The code that loads theInvoice
constant is thread safe (see load_missing_constant and require_or_load methods) and loads theInvoice
constant only once. TheInvoice
constant is then saved in the constants table of one of createdBilling
modules, but step 2.1.2 is invoked for each of them resulting in theLoadError
error being raised.It seems that the issue can be solved by simply making
autoload_module!
method thread safe. I’ve experimented with this solution and it appears to have resolved the issue.I had to disable wrap_parameters as a workaround