parallel: PG Error: PG::UnableToSend

Given this, here’s the way I’ve dealt with using PSQL databases with multiple processes: (works perfectly)

def reconnect_database
  config = ActiveRecord::Base.connection.instance_variable_get("@config").stringify_keys!
  ActiveRecord::Base.establish_connection(config)
end

ActiveRecord::Base.connection.disconnect!
(1..process_number).each do |process_index|
  processes << Process.fork do
    reconnect_database
    # Do stuff
  end
end
Process.waitall
reconnect_database

Okay, I didn’t expect this:

I tried the last file, but with mysql instead, and it fails… Even though similar statements work just peachy in my full-stack Rails app. Maybe there’s another component at play.

mysql_ar_parallel_test.rb

#!/usr/bin/env ruby

require 'mysql2'
require 'active_record'
require 'parallel'

$stderr.puts '---',
    RUBY_DESCRIPTION,
    "MySQL version: #{}",
    "ActiveRecord version: #{ActiveRecord.version}",
    "Parallel version: #{Parallel::VERSION}",
    '---'

class Person < ActiveRecord::Base
  establish_connection(
    adapter:  "mysql2",
    host:     "localhost",
    username: "test",
    password: "test",
    database: "test"
  )
end

puts "Non Threaded:"
puts "Expect: 1"
puts Person.first.id


puts "Parallel Gem:"
puts "Expect 1,2,3 in any order"

Parallel.each(Person.limit(3), in_processes: 3) do |person|
  puts person.id
end

puts "Repeat Non Threaded:"
puts "Expect: 1"
puts Person.first.id

Output:

---
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
MySQL version: 
ActiveRecord version: 4.2.5.1
Parallel version: 1.6.2
---
Non Threaded:
Expect: 1
1
Parallel Gem:
Expect 1,2,3 in any order
12

3
Repeat Non Threaded:
Expect: 1
/Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/mysql2-0.4.3/lib/mysql2/client.rb:107:in `_query': Mysql2::Error: MySQL server has gone away: SELECT  `people`.* FROM `people`  ORDER BY `people`.`id` ASC LIMIT 1 (ActiveRecord::StatementInvalid)
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/mysql2-0.4.3/lib/mysql2/client.rb:107:in `block in query'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/mysql2-0.4.3/lib/mysql2/client.rb:106:in `handle_interrupt'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/mysql2-0.4.3/lib/mysql2/client.rb:106:in `query'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `block in execute'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract_adapter.rb:472:in `block in log'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activesupport-4.2.5.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract_adapter.rb:466:in `log'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `execute'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/mysql2_adapter.rb:235:in `exec_query'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract/database_statements.rb:355:in `select'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/querying.rb:39:in `find_by_sql'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation.rb:639:in `exec_queries'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation.rb:515:in `load'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation.rb:243:in `to_a'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation/finder_methods.rb:500:in `find_nth_with_limit'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation/finder_methods.rb:484:in `find_nth'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/relation/finder_methods.rb:127:in `first'
    from /Users/duffyjp/.rvm/gems/ruby-2.2.3/gems/activerecord-4.2.5.1/lib/active_record/querying.rb:3:in `first'
    from pg_ar_parallel_mysql_test.rb:39:in `<main>'

Edit:

Workaround # 1 does work for this

Parallel.each(Person.limit(3), in_processes: 3) do |person|
  puts person.id
end
ActiveRecord::Base.connection.reconnect!

I’ll try in Postgres this afternoon.