LightGBM: [R-package] Installation with R4.0.0 on Windows is broken

Summary

For people arriving from a search engine, if you have R 4.0.x, are trying to build the R package on Windows, and get this error:

R version passed into FindLibR.cmake: 4.0.0
Creating R.lib and R.def
CMake Error at cmake/modules/FindLibR.cmake:48 (message):
Either gendef.exe or dlltool.exe not found!

Do you have Rtools installed with its MinGW's bin/ in PATH?
Call Stack (most recent call first):
cmake/modules/FindLibR.cmake:204 (create_rlib_for_msvc)
CMakeLists.txt:76 (find_package)

First, please put a 👍 on this comment so we can get a sense of how many people are facingt the problem.

You have two options for now

1. Open R-package/src/install.libs.R and set use_mingw <- TRUE. This will result in a package where training might be slower, but it should work because MinGW compilers are bundled with all versions of RTools.
2. Try @mayer79 's suggestion from https://github.com/microsoft/LightGBM/issues/3064#issuecomment-626173049...install RTools 3.5.0, then put gendef.exe from there into the bin/ directory of your installation of RTools 4.0
   • You can get old versions of RTools from https://cran.r-project.org/. Click “Download R for Windows”, then “Rtools”, then “this page” to get old versions

UPDATE: try the fix in https://github.com/microsoft/LightGBM/issues/3064#issuecomment-626273259 until #3065 is merged

The rest of the information below is for those who want to discuss why this is broken and how it will get permanently fixed. If you just want to get LightGBM’s R package working, you can ignore it and trust a better solution will be coming in the next few days and documented in our official installation instructions for R.

I just created #3065 , which introduces a possible fix! It will take some time to get it ready to be merged (especially since I want to wait for #2965 to be merged so we can test with Visual Studio in CI), but if you want to try it you can run the following:

git clone https://github.com/jameslamb/LightGBM.git
cd LightGBM
git fetch origin fix/r-4.0
git checkout fix/r-4.0

# install LightGBM
Rscript build_r.R

# test that it worked
cd R-package/tests
Rscript testthat.R

^ if anyone arriving at this issue tries that, please let me know how it goes! I updated to R 4.0 and Rtools 4.0 on my machine tonight and this is working for me.

To install gendef and mingw32-make.exe, open the rtools40 shell and type:

 pacman -S mingw-w64-{i686,x86_64}-{tools,make}

Thanks @jeroen for your help here and your amazing work on the R toolchain for Windows!

I think we will move ahead with my proposal in #3065 , using objdump to generate a .def file and make.exe for building the package, since those tools are bundled in previous Rtools and Rtools40.

I really want to preserve the user experience of our installation “just working” if you have CMake and Rtools.

This worked for me also, you are great life saver, I am struggling this for last 2 days

Thank you @jameslamb , I am sure many R users will appreciate this!

James, I tested with use_mingw <- TRUE but there is no mingw32-make.exe in C:\rtools40\mingw64\bin (the old rtools had it here). rtools40 is not frozen yet, so I am actually not sure if this is a version undergoing changes or not. Thanks so much for looking into this.

😱 ok thanks! I am going to be looking into this later today. I thought from https://github.com/r-windows/docs/blob/master/faq.md that all the mingw stuff that was in RTools 3.5 would be in 4.0, but maybe I made too many assumptions.

I’ve been on R3.6.x in my personal dev environment (waiting for R 4.0 and RTools 4.0 to be out a bit longer), so I haven’t hit this. Really appreciate your report and the information you gave to help us reproduce it.

Worst-case, we can host gendef.exe ourselves and have the install script download it. That’s not ideal but it’s also not too terrible, I think.

By the way we are working hard in parallel on getting the package to CRAN to alleviate some of this pain.

James, I tested with use_mingw <- TRUE but there is no mingw32-make.exe in C:\rtools40\mingw64\bin (the old rtools had it here). rtools40 is not frozen yet, so I am actually not sure if this is a version undergoing changes or not. Thanks a lot for caring about such installation issues. Very highly appreciated!

Details

Thank you for using LightGBM and for the reports @mayer79, @gmobaz, and @RaminZi ! We don’t yet test against R 4.0 (#3024 ) so I was worried this might happen.

I manually copied gendef.exe from the old RTools bundle and *magic, magic, installation worked! Do we need to contact Rtools maintainers? Or is it just me?

Thanks for trying this! It definitely makes sense that if gendef.exe is not bundled with RTools anymore, installs of the R package using Visual Studio compilers will not work. Similar to xgboost’s R package, we use gendef.exe to build a .lib file that can be linked into the LighGBM library. The relevant code is here, and the documentation on why this is necessary for Visual Studio projects is here

I will look into this in the next day or two and contact CRAN about it if necessary.

If you have time, I would appreciate if you could try my first suggestion in https://github.com/microsoft/LightGBM/issues/3064#issuecomment-626215933 (mingw <- TRUE). I know it’s not ideal and I’m committed to getting Visual Studio builds fixed soon, but if you’re able to tell me how that goes it will help me understand if there are other R4.0-specific issues with the LightGBM R package.

I manually copied gendef.exe from the old RTools bundle and *magic, magic, installation worked! Do we need to contact Rtools maintainers? Or is it just me?