ruby-argon2: Fails to load with Rubygems 3.4

I’ve installed ruby 3.2 on macOS Ventura (13.0.1) with M1 (arm64) using ruby-build. Launching irb and typing require "argon2" I see the following:

/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/ffi-1.15.5/lib/ffi/library.rb:145:in `block in ffi_lib': Could not open library '/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle': dlopen(/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle, 0x0005): tried: '/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle' (unloadable mach-o file type 10 '/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle'), '/System/Volumes/Preboot/Cryptexes/OS/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle' (no such file), '/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/libargon2_wrap.bundle.dSYM/Contents/Resources/DWARF/libargon2_wrap.bundle' (unloadable mach-o file type 10 '/Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/arg (LoadError)
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/ffi-1.15.5/lib/ffi/library.rb:99:in `map'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/ffi-1.15.5/lib/ffi/library.rb:99:in `ffi_lib'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:10:in `<module:Ext>'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:8:in `<module:Argon2>'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:6:in `<top (required)>'
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/lib/argon2.rb:4:in `<top (required)>'
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `require'
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:160:in `rescue in require'
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:149:in `require'
	from (irb):2:in `<main>'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'
	from /Users/dave/.rbenv/versions/3.2.0/bin/irb:25:in `load'
	from /Users/dave/.rbenv/versions/3.2.0/bin/irb:25:in `<main>'
<internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- argon2 (LoadError)
	from <internal:/Users/dave/.rbenv/versions/3.2.0/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
	from (irb):2:in `<main>'
	from /Users/dave/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'
	from /Users/dave/.rbenv/versions/3.2.0/bin/irb:25:in `load'
	from /Users/dave/.rbenv/versions/3.2.0/bin/irb:25:in `<main>'

Running on CI with Github Actions and ruby 3.2 on Ubuntu 22.04.1 (x86_64 arch) I see a similar but different error:

LoadError: cannot find 'argon2_wrap' library
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/ffi-compiler-1.0.1/lib/ffi-compiler/loader.rb:21:in `find'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:10:in `<module:Ext>'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:8:in `<module:Argon2>'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/argon2-2.1.2/lib/argon2/ffi_engine.rb:6:in `<main>'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/argon2-2.1.2/lib/argon2.rb:4:in `<main>'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/bootsnap-1.15.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
/home/runner/work/my-project-name/my-project-name/config/application.rb:24:in `<top (required)>'
/home/runner/work/my-project-name/my-project-name/config/environment.rb:2:in `require_relative'
/home/runner/work/my-project-name/my-project-name/config/environment.rb:2:in `<top (required)>'
/home/runner/work/my-project-name/my-project-name/Rakefile:3:in `require_relative'
/home/runner/work/my-project-name/my-project-name/Rakefile:3:in `block in <top (required)>'
/home/runner/work/my-project-name/my-project-name/vendor/bundle/ruby/3.2.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/hostedtoolcache/Ruby/3.2.0/x64/bin/bundle:25:in `load'
/opt/hostedtoolcache/Ruby/3.2.0/x64/bin/bundle:25:in `<main>'
Tasks: TOP => assets:precompile => environment
(See full trace by running task with --trace)

I tried cloning the argon2 repo, and was able to run bin/setup, bin/test, and rake test all without any errors. I looked at https://github.com/technion/ruby-argon2/blob/master/test/test_helper.rb#L3 and found this helped. Steps to reproduce:

  1. Open irb session in the ruby-argon2 directory
  2. Type require "argon2" see failures (same as above for macOS)
  3. Type $LOAD_PATH.unshift File.expand_path("lib")
  4. Type require "argon2" and now it works.

Note the above only works after running bin/setup, even though I’ve installed argon2 already. This seems to be because when running bin/setup, the correct file in ext/argon2_wrap is created, specifically the libargon_wrap2 file:

On macOS, Ruby 3.1.3 (argon2 loads no issue)

~ ❯❯❯ ls -la .rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/argon2-2.1.2/ext/argon2_wrap/
total 496
drwxr-xr-x  9 dave  staff     288 Dec  5 16:58 .
drwxr-xr-x  4 dave  staff     128 Dec  5 16:58 ..
-rw-r--r--  1 dave  staff    1958 Dec  5 16:58 Makefile
-rw-r--r--  1 dave  staff    4504 Dec  5 16:58 argon_wrap.c
-rw-r--r--  1 dave  staff      98 Dec  5 16:58 extconf.rb
-rwxr-xr-x  1 dave  staff   76450 Dec  5 16:58 libargon2_wrap.bundle
drwxr-xr-x  3 dave  staff      96 Dec  5 16:58 libargon2_wrap.bundle.dSYM
-rwxr-xr-x  1 dave  staff  151552 Dec  5 16:58 libargon2_wrap.so
-rw-r--r--  1 dave  staff    4500 Dec  5 16:58 test.c

On macOS, Ruby 3.2.0 (missing the libargon2_wrap.bundle file, argon2 errors on require)

~ ❯❯❯ ls -la .rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/
total 344
drwxr-xr-x  8 dave  staff     256 Dec 25 12:14 .
drwxr-xr-x  4 dave  staff     128 Dec 25 12:14 ..
-rw-r--r--  1 dave  staff    1958 Dec 25 12:14 Makefile
-rw-r--r--  1 dave  staff    4504 Dec 25 12:14 argon_wrap.c
-rw-r--r--  1 dave  staff      98 Dec 25 12:14 extconf.rb
drwxr-xr-x  3 dave  staff      96 Dec 25 12:14 libargon2_wrap.bundle.dSYM
-rwxr-xr-x  1 dave  staff  151552 Dec 25 12:14 libargon2_wrap.so
-rw-r--r--  1 dave  staff    4500 Dec 25 12:14 test.c

On Ubuntu, Ruby 3.1.3 (argon2 loads no issue)

$ ls -la .rbenv/versions/3.1.3/lib/ruby/gems/3.1.0/gems/argon2-2.1.2/ext/argon2_wrap/
total 256
drwxrwxr-x 2 ubuntu ubuntu   4096 Dec 25 18:10 .
drwxrwxr-x 4 ubuntu ubuntu   4096 Dec 25 18:10 ..
-rw-r--r-- 1 ubuntu ubuntu   1958 Dec 25 18:10 Makefile
-rw-r--r-- 1 ubuntu ubuntu   4504 Dec 25 18:10 argon_wrap.c
-rw-r--r-- 1 ubuntu ubuntu     98 Dec 25 18:10 extconf.rb
-rwxrwxr-x 1 ubuntu ubuntu 227056 Dec 25 18:10 libargon2_wrap.so
-rw-r--r-- 1 ubuntu ubuntu   4500 Dec 25 18:10 test.c

On Ubuntu, Ruby 3.2.0 (missing the libargon2_wrap.so file, argon2 errors on require)

$ ls -la .rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.2/ext/argon2_wrap/
total 32
drwxrwxr-x 2 ubuntu ubuntu 4096 Dec 25 17:49 .
drwxrwxr-x 4 ubuntu ubuntu 4096 Dec 25 17:49 ..
-rw-r--r-- 1 ubuntu ubuntu 1958 Dec 25 17:49 Makefile
-rw-r--r-- 1 ubuntu ubuntu 4504 Dec 25 17:49 argon_wrap.c
-rw-r--r-- 1 ubuntu ubuntu   98 Dec 25 17:49 extconf.rb
-rw-r--r-- 1 ubuntu ubuntu 4500 Dec 25 17:49 test.c

I’m not sure if this is an ffi issue, or the way argon2 is using ffi.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 36 (16 by maintainers)

Commits related to this issue

Most upvoted comments

Version 2.2.0 has just been pushed. Let’s cross fingers this is behind us.

+5
technion on Dec 27, 2022

Thanks for your patience and help everyone.

I’ve just pushed a rewrite of the Makefile and it appears to work.

root@7ffc7d633108:/ruby-argon2# ruby -v
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
root@7ffc7d633108:/ruby-argon2# gem install argon2-2.1.3.gem
Building native extensions. This could take a while...
Successfully installed argon2-2.1.3
Parsing documentation for argon2-2.1.3
Installing ri documentation for argon2-2.1.3
Done installing documentation for argon2 after 2 seconds
1 gem installed

irb(main):001:0> require 'argon2'
=> true
irb(main):002:0> hasher = Argon2::Password.new
irb(main):003:0> hasher.create("password")
=> "$argon2id$v=19$m=65536,t=2,p=1$yt9iCVsmXYkWmkJ4YqhYaw$/74dvohSpLR/ix7iUG0M1EugxazeYm2y5OKPg2luY5Y"

It does however… cause the test suite to fail. Working on that next.

+2
technion on Dec 26, 2022

I have tested some more and I see that this broke between rubygems-update version 3.3.26 and 3.4.0. The first one works, the latter one breaks.

+2
communiteq on Dec 26, 2022

For me 2.2.0 resolves the issue!!! Thank you @technion ❤️

+1
communiteq on Dec 27, 2022

@hsbt I’ve come up with a substantively better workaround, removed the “system” call and just added a cp to my Makefile to move the library something your new Rubygems doesn’t clean up. If anyone would like to try the updated https://ctadvisor.lolware.net//argon2-2.2.0.gem it would be appreciated, I’m hoping we have a proper solution finally.

+1
technion on Dec 27, 2022

Commit 927bf0e reverts everything back to my original Makefile, it came to my attention there was even more platform dependent code that was going to need porting and I was slowly losing my mind over it.

I’ve introduced a horrible workaround to this whole issue, in the form of adding this line to extconf.rb:

system "make"

I hate it, but here we are, the CI works. I’ve created a package because testing this gem from github has its own challenges with the submodule.

root@780092a29cb2:/# ruby -v
ruby 3.2.0rc1 (2022-12-22) [x86_64-linux]
root@780092a29cb2:/# wget https://ctadvisor.lolware.net//argon2-2.2.0.gem
root@780092a29cb2:/# gem install argon2-2.2.0.gem
Building native extensions. This could take a while...
Successfully installed argon2-2.2.0
1 gem installed
root@780092a29cb2:/# irb
irb(main):001:0> require 'argon2'
=> true
irb(main):002:0> hasher = Argon2::Password.new(t_cost: 2, m_cost: 16, p_cost: 1)
irb(main):003:0> hasher.create("password")
=> "$argon2id$v=19$m=65536,t=2,p=1$CP2ii3RBHWecd+XhNOcyNw$Lka/44bkEuaGX2/8FFR+7axu4tDcph8jrvrX+ACXwJ0"
irb(main):004:0>

If anyone would like to test the above package it would be appreciated. I’m just cleaning the Rubocop errors and should be able to tag a release soon.

I think this confirms the issue is not ffi-loader’s resolution, it’s when you type “gem install” it simply never runs “make” like it should.

+1
technion on Dec 27, 2022

Just to confirm, it appears that although the CI works fine, there’s something still broken here. I had a fair bit of trouble due to the lack of a current Ruby 3.2 docker container but I’ve built a more complete lab of this.

root@7e9fcf33fcaf:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

# Install prereqs..

eval "$(/root/.rbenv/bin/rbenv init - bash)"

rbenv install 3.2.0
root@7e9fcf33fcaf:/# mkdir myapp
root@7e9fcf33fcaf:/# cd myapp
root@7e9fcf33fcaf:/myapp# bundle init
Writing new Gemfile to /myapp/Gemfile
root@7e9fcf33fcaf:/myapp# nano Gemfile
root@7e9fcf33fcaf:/myapp# cat Gemfile
# frozen_string_literal: true

source "https://rubygems.org"
gem "argon2", "2.1.3"
root@7e9fcf33fcaf:/myapp# bundle install
Don't run Bundler as root. Installing your bundle as root will break this application for all non-root users on this
machine.
Fetching gem metadata from https://rubygems.org/.
Using rake 13.0.6
Fetching ffi 1.15.5
Using bundler 2.4.1
Installing ffi 1.15.5 with native extensions
Fetching ffi-compiler 1.0.1
Installing ffi-compiler 1.0.1
Fetching argon2 2.1.3
Installing argon2 2.1.3 with native extensions
Bundle complete! 1 Gemfile dependency, 5 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
root@7e9fcf33fcaf:/myapp# bundle exec myapp.rb
bundler: failed to load command: myapp.rb (myapp.rb)
/root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/ffi-compiler-1.0.1/lib/ffi-compiler/loader.rb:21:in `find': cannot find 'argon2_wrap' library (LoadError)

I’ll continue investigating but I’m sure appreciative of any other suggestions. I do note that in this environment, it simply seems to never be running the actual build script.

openat(AT_FDCWD, "/root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap", O_RDONLY|O_CLOEXEC|O_DIRECTORY) = 5
openat(AT_FDCWD, "/root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap", O_RDONLY|O_CLOEXEC|O_DIRECTORY) = 5
write(2, "/root/.rbenv/versions/3.2.0/lib/"..., 2163/root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/ffi-compiler-1.0.1/lib/ffi-compiler/loader.rb:21:in `find': cannot find 'argon2_wrap' library (LoadError)
root@7e9fcf33fcaf:/myapp# ls /root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap
Makefile  argon_wrap.c  extconf.rb  test.c

This workaround takes care of it:

root@7e9fcf33fcaf:/myapp# cd /root/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap
root@7e9fcf33fcaf:~/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap# make
cc -pthread -O3 -Wall -g -I../phc-winner-argon2/include -I../phc-winner-argon2/src -shared -fPIC ../phc-winner-argon2/src/argon2.c ../phc-winner-argon2/src/core.c ../phc-winner-argon2/src/blake2/blake2b.c ../phc-winner-argon2/src/thread.c ../phc-winner-argon2/src/encoding.c argon_wrap.c ../phc-winner-argon2/src/ref.c -o libargon2_wrap.so
root@7e9fcf33fcaf:~/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/argon2-2.1.3/ext/argon2_wrap# cd
root@7e9fcf33fcaf:~# irb
irb(main):001:0> require 'argon2'
=> true
irb(main):002:0>
+1
technion on Dec 26, 2022

Thanks for reporting this and the detail. Much like the work in this thread, I’m struggling to work out whether this is an FFI issue or just something involving the build scripts. I’m working with the Ruby 3.2-rc1 Docker image (which currently seems to be the latest) and I can replicate it there.

I’m going through open issues on FFI and nothing seems relevant.

I’ve found if you make sure the library is built and do an strace, it does load it even before complaining it’s not built:

openat(AT_FDCWD, “/ruby-argon2/ext/argon2_wrap/libargon2_wrap.so”, O_RDONLY|O_CLOEXEC) = 5

Whatever’s going on I’m sure someone else’s gem will have the same issue, which I’m hoping leads to a fix.

@communiteq Just to confirm are you saying you had this same break without upgrading to Ruby 3.2.0? I can’t replicate that.

+1
technion on Dec 26, 2022
Read more comments on GitHub
← xgo: Unable to build for target linux/amd64.
nuki_hub: Cannot use let's encrypt certificate →