async: Sync gets stuck

We’re using async heavily in our project and while writing tests I noticed that some pieces of code consistently get stuck while using Sync with a Semaphore. I unfortunately do not have a minimal reproduction, but it consistently happens in the same parts of our code, only in tests. Here’s the line where it gets stuck:

begin
  @selector.select(interval)
rescue Errno::EINTR
  # Ignore.
end

here’s the stacktrace I was able to procure(and here’s how I procured it):

/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:317:in `select'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:317:in `run_once!'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:353:in `run_once'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:408:in `block in run'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:386:in `block in run_loop'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:383:in `handle_interrupt'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:383:in `run_loop'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/async/scheduler.rb:407:in `run'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/async-2.17.0/lib/kernel/sync.rb:28:in `Sync'
/Users/me/my_app/app/models/workflow.rb:187:in `block (2 levels) in execute_step'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:616:in `block in within_new_transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activesupport-7.2.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `handle_interrupt'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activesupport-7.2.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `block in synchronize'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activesupport-7.2.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `handle_interrupt'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activesupport-7.2.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `synchronize'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:613:in `within_new_transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:361:in `transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:234:in `block in transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:389:in `with_connection'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_handling.rb:296:in `with_connection'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:233:in `transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:354:in `transaction'
/Users/me/my_app/app/models/workflow.rb:186:in `block in execute_step'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:359:in `transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:234:in `block in transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:389:in `with_connection'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/connection_handling.rb:296:in `with_connection'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:233:in `transaction'
/Users/me/.asdf/installs/ruby/3.3.5/lib/ruby/gems/3.3.0/gems/activerecord-7.2.1/lib/active_record/transactions.rb:354:in `transaction'

more context:

  • it’s a sidekiq job
  • we’re using rspec
  • we’re using rails(as you can see on the stacktrace)

I have no idea how to provide more context. Let me know what I can do

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (9 by maintainers)

Most upvoted comments

It does indeed sound like something caused by transactional fixtures. By default in tests only one connection is used and if you spawn another fiber/thread, when it check out a connection it will be handled the same one as the main thread.

This is so it’s querying inside the same transaction. But yes, in some cases it can cause deadlock if you stray a bit too far from the main path, but I’m not sure if it’s something we can really fix here.

You can confirm this by disabling transactional fixtures for that test: https://guides.rubyonrails.org/testing.html#opting-out-of-test-transactions

that is it! Worked without transactional fixtures. Thanks so much for figuring it out guys 👍

Okay, I’ll try it, thanks for the repro!

sorry for not replying, was occupied with work. Hope you get better soon! In the meanwhile, I have managed to reproduce this in a blank rails app. Here’s what I did: 1.

rails new test-async
cd test-async
bundle add async
rails g model dummy
rails db:create db:migrate
  1. add these to the application.rb:
    config.active_support.isolation_level = :fiber
    config.active_record.permanent_connection_checkout = :disallowed
  1. add this to dummy_test.rb:
  test "hangs" do
    step = proc {
      Sync do
        ApplicationRecord.with_connection
      end
    }

    dummy = Dummy.create!
    dummy.transaction do
      dummy.instance_exec(&step)
    end
  end

now, I couldn’t really print_hierarchy cause it just hangs, maybe I don’t really understand how to do this, sorry. My current observation is that it’s not necessarily async, it’s probably rails and async and the way they work together. Let me know if I should post this as a rails issue instead. Also do let me know if more info is needed, I’ll try to help out