rails: LoadError when multiple threads try to load the same namespaced class
Version Information
Ruby: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17]
Rails: Rails 5.2.0
Steps to reproduce
- Create brand new rails project using
rails new awesome_application - Create a directory called
app/models/deepand a file calledapp/models/deep/beep.rbwith the following beautifully succinct code:
module Deep
Class Beep
end
end
- Open up the rails console and run
Deep::Beep- all good! - Close and open up the rails console again and run
2.times { |i| Thread.new { Rails.application.executor.wrap { Deep::Beep } } }- oh no! Sometimes fails with:
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: already initialized constant Deep
.../activesupport-5.2.0/lib/active_support/dependencies.rb:452: warning: previous definition of Deep was here
#<Thread:0x00007fd4795428b0@(irb):1 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
5: from (irb):1:in `block (2 levels) in irb_binding'
4: from .../activesupport-5.2.0/lib/active_support/execution_wrapper.rb:87:in `wrap'
3: from (irb):1:in `block (3 levels) in irb_binding'
2: from .../activesupport-5.2.0/lib/active_support/dependencies.rb:193:in `const_missing'
1: from .../bootsnap-1.3.0/lib/bootsnap/load_path_cache/core_ext/active_support.rb:43:in `load_missing_constant'
.../activesupport-5.2.0/lib/active_support/dependencies.rb:510:in `load_missing_constant': Unable to autoload constant Deep::Beep, expected .../app/models/deep/beep.rb to define it (LoadError)
Why does this happen?
There is a race condition where two threads will concurrently run the const_missing method for the Deep module, and both will run the autoload_module! method and create two separate Deep module objects on each thread - I believe it should be a single object for any given module.
Then, one of the threads will load the Beep class into its local module object using require_or_load (which is mutexed globally with Dependencies.load_interlock) and continue happily, while the other will run require_or_load, determine that Beep is already loaded, and then raise LoadError because its local module (from_mod in the source code) does not have the Beep class defined on it.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 17
- Comments: 33 (3 by maintainers)
An other workaround that works for us was to only use one thread in development to keep the
eager_load = false.Ha! Yes, you are correct. That’s what I get for trying to be funny - my actual tests were using
AandA::B, which is harder to mess up 😃The issue still stands, so I updated the original post, thanks.
I’ve encountered the same issue on the project I’m working on. After some investigation I found the following. If we take this code as an example
The process of loading
Billing::Invoiceconstant can be described by following pseudocode:Billing(it is not resolved by ruby so it has to be autoloaded) 1.1. check if filebilling.rbexists in autoload paths 1.2. check if directorybillingexists in autoload paths (it does) 1.2.1. create moduleBillingInvoice2.1. check if filebilling/invoice.rbexists in autoload paths (it does) 2.1.1. load/require the filebilling/invoice.rb2.1.2. check if the constantInvoiceis now defined on theBillingmodule and raiseLoadErrorerror if notThe issue with concurrent execution arises on step 1.2.1 as the code which creates “automatic” modules is not thread safe. See
autoload_module!methodRails does not check if the module is already defined and does not synchronise access to this method which in turn results in multiple instances of
Billingmodule being created. On each of those instances theconst_missingmethod is later invoked (in step 2) for the constantInvoice. The code that loads theInvoiceconstant is thread safe (see load_missing_constant and require_or_load methods) and loads theInvoiceconstant only once. TheInvoiceconstant is then saved in the constants table of one of createdBillingmodules, but step 2.1.2 is invoked for each of them resulting in theLoadErrorerror being raised.It seems that the issue can be solved by simply making
autoload_module!method thread safe. I’ve experimented with this solution and it appears to have resolved the issue.I had to disable wrap_parameters as a workaround