extendr: Failed to generate wrapper functions when calling tokio runtime `block_on` in exposed struct constructor

Hi!

We are building a cross platform tool to work with compact URIs with a core in rust that compiles to many languages, including R: https://github.com/biopragmatics/curies.rs

We managed to get something working for R here: https://github.com/biopragmatics/curies.rs/blob/add-r-bindings/r/src/rust/src/lib.rs

It can be called this way from R:

library(curiesr)
converter <- ConverterR$new()
curie <- converter$compress("http://purl.obolibrary.org/obo/DOID_1234")
print(curie)

But as we try to instantiate our Converter struct from a document available on the internet as a URL, we are calling an async function.

And to make this function blocking we use the tokio runtime, but as soon as I add the init_converter() function (which call the runtime) to my Converter new() constructor I get the cryptic error Error in value[[3L]](): Failed to generate wrapper functions

fn init_converter() -> Converter {
    let rt = Runtime::new().unwrap();
    rt.block_on(async { get_bioregistry_converter().await.unwrap() })
}

Removing the init_converter to synchronously build the converter: it works like a charm

Which seems to point at the tokio runtime as the culprit for generating this error at wrap time!

Do you have any idea how we could overcome this? I could not find many issues related to async and runtimes in your repo, so maybe we are doing something wrong on our side? Maybe we should use a futures runtime instead of tokio?

About this issue

Original URL
State: closed
Created 5 months ago
Comments: 16 (10 by maintainers)

Most upvoted comments

Alrighty then

PKG_LIBS = -L$(LIBDIR) -lcuriesr -lssl

in Makevars solves it for me. I got this idea from this https://stackoverflow.com/a/73898812/63696

Let us know what happens on your end.

CGMossa on Feb 11, 2024

All right @CGMossa ! It works like a charm with this flag, thanks a lot for the debugging!

✔ Writing 'R/extendr-wrappers.R'
ℹ Updating curiesr documentation
Setting `RoxygenNote` to "7.3.1"
ℹ Loading curiesr
✖ extendr-wrappers.R:12: `@docType "package"` is deprecated.
ℹ Please document "_PACKAGE" instead.
Writing curiesr-package.Rd
> library(curiesr)
converter <- ConverterR$new()
curie <- converter$compress("http://purl.obolibrary.org/obo/DOID_1234")
print(curie)
[1] "doid:1234"

I guess we can close this issue then 😃

vemonet on Feb 11, 2024

I just tried to recreate this in my WSL container.

I still believe you should update your local Cargo.lock file with cargo update, but:

─  DONE (curiesr)
Error in `value[[3L]]()`:
! Failed to generate wrapper functions.
✖ in callr subprocess.
Run `rlang::last_trace()` to see where the error occurred.

And in rextendr, the code that fails is this:

debug: tryCatch(make_wrappers_externally(module_name = as_valid_rust_name(pkg_name),
    package_name = pkg_name, outfile = outfile, path = path,
    use_symbols = TRUE, quiet = quiet), error = function(e) {
    cli::cli_abort(c("Failed to generate wrapper functions.",
        x = e[["message"]]), class = "rextendr_error")
})

The way these wrappers are generated is very clever. Since Rust knows about its code, then from the Rust code we provide a function, that outputs the wrappers. Then rextendr calls that function! But we don’t want that loading, to affect the loading of symbols onto the current session, so we have a sub-r-process that loads the Rust library, and calls a make-wrappers function in it.

To debug this (for us, not you 😊), we could do the following:

options(callr.traceback = TRUE)
rs <- callr::r_session$new()
rs$run(\() rextendr::document(quiet=FALSE))
# see error...
rs$debug()

And what happens here is a I get

Debugging in process 59817, press CTRL+C (ESC) or type .q to quit. Commands:
  .where       -- print stack trace
  .inspect <n> -- inspect a frame, 0 resets to .GlobalEnv
  .help        -- print this message
  .q           -- quit debugger
  <cmd>        -- run <cmd> in frame or .GlobalEnv

9: signal_abort(cnd, .file)
8: rlang::abort(message, ..., call = call, use_cli_format = TRUE,
       .frame = .frame)
7: cli::cli_abort(c("Failed to generate wrapper functions.", x = e[["message"]]),
       class = "rextendr_error")
6: value[[3L]](cond)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
4: tryCatchList(expr, classes, parentenv, handlers)
3: tryCatch(make_wrappers_externally(module_name = as_valid_rust_name(pkg_name),
       package_name = pkg_name, outfile = outfile, path = path,
       use_symbols = TRUE, quiet = quiet), error = function(e) {
       cli::cli_abort(c("Failed to generate wrapper functions.",
           x = e[["message"]]), class = "rextendr_error")
   })
2: register_extendr(path = pkg, quiet = quiet)
1: rextendr::document(quiet = FALSE)

I then press .inspect 5

RS 59817 > .inspect 5

Looking and cond

RS 59817 (frame 5) > cond
<callr_error/rlib_error_3_0/rlib_error/error>
Error:
! in callr subprocess.
Caused by error in `dyn.load(library_path)`:
! unable to load shared object '/home/mini/curies.rs/r/src/curiesr.so':
  /home/mini/curies.rs/r/src/curiesr.so: undefined symbol: SSL_get_error

And there you have it there is a missing symbol. It means a static/shared library isn’t being linked to in the build process, that make up what the program requires. But now, this is on the Linux side.

SSL_get_error

CGMossa on Feb 11, 2024

Right, what does rlang::last_trace() say? Also, what about rextendr::rust_siterep()?

CGMossa on Feb 9, 2024

I am getting this output:

! Installed rextendr is older than the version used with this packageYou have "0.3.1" but you need "0.3.1.9000"
ℹ Generating extendr wrapper functions for package: curiesr.
Error in `value[[3L]]()`:
! Failed to generate wrapper functions.
✖ in callr subprocess.
Run `rlang::last_trace()` to see where the error occurred.

I am trying installing from github:

remotes::install_github("extendr/rextendr")

Now getting the output:

ℹ Generating extendr wrapper functions for package: curiesr.
Error in `value[[3L]]()`:
! Failed to generate wrapper functions.
✖ in callr subprocess.
Run `rlang::last_trace()` to see where the error occurred.

vemonet on Feb 9, 2024

Cool! Thanks, that means it should work, there is just something going on with my setup

I am running this on Ubuntu 22.04, maybe I am missing some linux related dependencies?

I have installed the following R packages:

install.packages("usethis")
install.packages("rextendr")
install.packages("devtools")
install.packages("testthat")

It was failing with R version 4.1.2, and I think rextendr was 0.3.1

I upgraded to R version 4.3.2, now reinstalling all the packages… takes time apparently

vemonet on Feb 9, 2024

Hmm, I pulled it and tried, but I couldn’t reproduce it. (I had to add -lcrypt32 -lsecur32 to Makevars.win, but I guess this is unrelated to your error). Did you try updateing rextendr?

> rextendr::document("r")
✔ Saving changes in the open files.
• The SystemRequirements field in the DESCRIPTION file is already set.
• Please update it manually if needed.
ℹ Generating extendr wrapper functions for package: curiesr.
ℹ Re-compiling curiesr (debug build)
── R CMD INSTALL ────────────────────────────────────────────────────────────────────────────────────────────
─  installing *source* package 'curiesr' ...
   ** using staged installation
   ** libs
   using C compiler: 'gcc.exe (GCC) 12.3.0'
   rm -Rf curiesr.dll ./rust/target/x86_64-pc-windows-gnu/release/libcuriesr.a entrypoint.o
   gcc  -I"C:/PROGRA~1/R/R-devel/include" -DNDEBUG     -I"C:/rtools43/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall -gdwarf-2 -mfpmath=sse -msse2 -mstackrealign  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always -c entrypoint.c -o entrypoint.o
   mkdir -p ./rust/target/libgcc_mock
   # `rustc` adds `-lgcc_eh` flags to the compiler, but Rtools' GCC doesn't have
   # `libgcc_eh` due to the compilation settings. So, in order to please the
   # compiler, we need to add empty `libgcc_eh` to the library search paths.
   #
   # For more details, please refer to
   # https://github.com/r-windows/rtools-packages/blob/2407b23f1e0925bbb20a4162c963600105236318/mingw-w64-gcc/PKGBUILD#L313-L316
   touch ./rust/target/libgcc_mock/libgcc_eh.a
   # CARGO_LINKER is provided in Makevars.ucrt for R >= 4.2
   if [ "true" != "true" ]; then \
   	export CARGO_HOME=/c/Users/Yutani/Documents/GitHub/curies.rs/r/src/.cargo; \
   fi && \
   	export CARGO_TARGET_X86_64_PC_WINDOWS_GNU_LINKER="x86_64-w64-mingw32.static.posix-gcc.exe" && \
   	export LIBRARY_PATH="${LIBRARY_PATH};/c/Users/Yutani/Documents/GitHub/curies.rs/r/src/./rust/target/libgcc_mock" && \
   	cargo build --target=x86_64-pc-windows-gnu --lib --release --manifest-path=./rust/Cargo.toml --target-dir ./rust/target
      Compiling curiesr v0.3.0 (C:\Users\Yutani\Documents\GitHub\curies.rs\r\src\rust)
   warning: unused import: `std::collections::HashSet`
    --> r\src\rust\src\lib.rs:1:5
     |
   1 | use std::collections::HashSet;
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(unused_imports)]` on by default
   
   warning: unused import: `curies::Record`
    --> r\src\rust\src\lib.rs:3:5
     |
   3 | use curies::Record;
     |     ^^^^^^^^^^^^^^
   
   warning: `curiesr` (lib) generated 2 warnings (run `cargo fix --lib -p curiesr` to apply 2 suggestions)
       Finished release [optimized] target(s) in 51.93s
   if [ "true" != "true" ]; then \
   	rm -Rf /c/Users/Yutani/Documents/GitHub/curies.rs/r/src/.cargo && \
   	rm -Rf ./rust/target/x86_64-pc-windows-gnu/release/build; \
   fi
   gcc -shared -static-libgcc -o curiesr.dll curiesr-win.def entrypoint.o -L./rust/target/x86_64-pc-windows-gnu/release -lcuriesr -lws2_32 -ladvapi32 -luserenv -lbcrypt -lntdll -lcrypt32 -lsecur32 -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-devel/bin/x64 -lR
   installing to C:/Users/Yutani/AppData/Local/Temp/RtmpeyrJ5D/devtools_install_35f04202478b/00LOCK-r/00new/curiesr/libs/x64
─  DONE (curiesr)
✔ Writing 'R/extendr-wrappers.R'
ℹ Updating curiesr documentation
Setting `RoxygenNote` to "7.3.1"
ℹ Loading curiesr
✖ extendr-wrappers.R:12: `@docType "package"` is deprecated.
ℹ Please document "_PACKAGE" instead.

yutannihilation on Feb 9, 2024