rails: Rails 7.0.3 vs Rails 6.1.4.4 regression when using shards in production

Steps to reproduce

# I'm using sharding for some models.
module Cmdmary
  # base class for AR
  class CmdmaryRecord < ApplicationRecord
    self.abstract_class = true

    connects_to shards: {
      fr: { writing: :comdev_fr },
      de: { writing: :comdev_de }
    }
  end
end

database.yml contain:
default: &default
  adapter: oracle_enhanced
  pool: 5
  timeout: 5000
development:
  primary:
    <<: *default
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx
  comdev_fr:
    <<: *default
    migrations_paths: db/migrate_none
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx
  comdev_de:
    <<: *default
    migrations_paths: db/migrate_none
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx
production:
  primary:
    <<: *default
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx
  comdev_fr:
    <<: *default
    migrations_paths: db/migrate_none
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx
  comdev_de:
    <<: *default
    migrations_paths: db/migrate_none
    database: sk-oracle11.sk.loc/XE
    username: xx
    password: xx

in the controller the connection is managed by:
        def handle_tenant
          ::Cmdmary::CmdmaryRecord.connected_to(shard: api_params[:tenant].to_sym, role: :writing) do
            Rails.logger.warn { "Tenant used:'#{api_params[:tenant]}', api:'#{api}'" }
            yield
          end
        end
      around_action :handle_tenant

Expected behavior

in rails 7.0.3 Rails should boot without error in production mode

Actual behavior

the rails server crash with:

SECRET_KEY_BASE=xx DEVISE_SECRET_KEY=xx RAILS_SERVE_STATIC_FILES=1 bundle exec rails server -b 0.0.0.0 -e production --log-to-stdout
=> Booting Puma
=> Rails 7.0.3 application starting in production
=> Run `bin/rails server --help` for more startup options
Exiting
/home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:309:in `connection_pool': ActiveRecord::ConnectionNotEstablished (ActiveRecord::ConnectionNotEstablished)
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:305:in `connection_db_config'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/activerecord-7.0.3/lib/active_record/type.rb:50:in `adapter_name_from'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/activerecord-7.0.3/lib/active_record/attributes.rb:216:in `attribute'
        from /var/www/dbpresenter-test/html/app/models/cmdmary/etudprix.rb:11:in `<class:Etudprix>'
        from /var/www/dbpresenter-test/html/app/models/cmdmary/etudprix.rb:5:in `<module:Cmdmary>'
        from /var/www/dbpresenter-test/html/app/models/cmdmary/etudprix.rb:3:in `<main>'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/kernel.rb:27:in `require'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/helpers.rb:127:in `const_get'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/helpers.rb:127:in `cget'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:239:in `block (2 levels) in eager_load'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/helpers.rb:41:in `block in ls'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/helpers.rb:27:in `each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader/helpers.rb:27:in `ls'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:234:in `block in eager_load'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:219:in `synchronize'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:219:in `eager_load'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:318:in `each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/zeitwerk-2.6.0/lib/zeitwerk/loader.rb:318:in `eager_load_all'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/application/finisher.rb:74:in `block in <module:Finisher>'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/initializable.rb:32:in `instance_exec'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/initializable.rb:32:in `run'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/initializable.rb:61:in `block in run_initializers'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:228:in `block in tsort_each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:431:in `each_strongly_connected_component_from'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:349:in `block in each_strongly_connected_component'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:347:in `each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:347:in `call'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:347:in `each_strongly_connected_component'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:226:in `tsort_each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/3.0.0/tsort.rb:205:in `tsort_each'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/initializable.rb:60:in `run_initializers'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/application.rb:372:in `initialize!'
        from /var/www/dbpresenter-test/html/config/environment.rb:7:in `<main>'
        from config.ru:5:in `require_relative'
        from config.ru:5:in `block in <main>'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/builder.rb:116:in `eval'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/builder.rb:116:in `new_from_string'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/builder.rb:105:in `load_file'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/builder.rb:66:in `parse_file'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/server.rb:349:in `build_app_and_options_from_config'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/server.rb:249:in `app'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/rack-2.2.3.1/lib/rack/server.rb:422:in `wrapped_app'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/commands/server/server_command.rb:76:in `log_to_stdout'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/commands/server/server_command.rb:36:in `start'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/commands/server/server_command.rb:143:in `block in perform'
        from <internal:kernel>:90:in `tap'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/commands/server/server_command.rb:134:in `perform'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/command/base.rb:87:in `perform'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/command.rb:48:in `invoke'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from /home/serge/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.12.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        from bin/rails:4:in `<main>'

System configuration

Rails 7.0.3:

Ruby 3.0.4:

In development mode it work in rails 7.0.3 ans in rails 6.1.4.4 In production mode it work in rails 6.1.4.4 and fail in rails 7.0.3

while debuging I found this in ActiveRecord::ConnectionAdapters def retrieve_connection_pool(owner, role: ActiveRecord::Base.current_role, shard: ActiveRecord::Base.current_shard) pool_config = get_pool_manager(owner)&.get_pool_config(role, shard) pool_config&.pool end

the shard value is :default so the result is nil

This method is used for all models at initialization time and the shard is not specified at this time (my code is not run) But in 6.1.4.4 I didn’t find this behavior during initialization

zeitwerk is used in the 2 cases and it fail with or without bootsnap gem.

Is this behavior normal or not ? this should be specified in the documentation

As a workaround I modified the model abstract class: connects_to shards: { fr: { writing: :comdev_fr }, de: { writing: :comdev_de }, default: { writing: :comdev_fr }, } and with this it’s working, but I’m not sure it’s the correct solution…

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 1
Comments: 23 (11 by maintainers)

Commits related to this issue

Set default_shard from connects_to hash If an application is using sharding, they may not want to use `default` as the `default_shard`. Unfortunately Rails expects there to be a shard named `default`... — committed to eileencodes/rails by eileencodes a year ago

Most upvoted comments

Hi @grobie I actually found a solution for this that doesn’t require me to refactor a ton of active record code. Since it is a slight behavior change I don’t think I’m going to backport it to 7.0. However, if you want to implement the same behavior as Rails for 7.0 you can set the default_shard in the Comdev model like this:

  class CmdmaryRecord < ApplicationRecord
    self.abstract_class = true
    self.default_shard = :fr

    connects_to shards: {
      fr: { writing: :comdev_fr },
      de: { writing: :comdev_de }
    }
  end

eileencodes on May 31, 2023

Would you accept a pull request to add this fallback to Type.adapter_name_from in order to make it independent of the presence of the default_shard?

No, I don’t think a fallback here is the right fix because this is indicative of a larger issue in Active Record and it’s reliance on Base.connection. I plan on fixing this a more correct way but it’s going to take some time. It requires redesigning parts of Active Record, which I’m in the process of doing. I will update this issue when there’s a PR solving this particular problem.

eileencodes on Sep 4, 2022