node-llama-cpp: NoBinaryFoundError for Windows in Electron when Upgrading from 3.0.0-beta44 -> 3.2.0

Issue description

NoBinaryFoundError for Windows when Upgrading from 3.0.0-beta44 -> 3.2.0

Expected Behavior

I’d expect the WIndows x64 prebuilt binary to be available for use so users do not have to build locally.

Actual Behavior

A NoBinaryFoundError is thrown when trying to start getLlama

Steps to reproduce

  • Use Electron + node-llama-cpp
  • build on windows
  • use getLlama()

Btw, prebuilt binary was found and worked great in 3.0.0-beta44

My Environment

Dependency Version
Operating System Win 10
CPU Intel I7
node-llama-cpp version 3.2.0

Additional Context

No response

Relevant Features Used

  • Metal support
  • CUDA support
  • Vulkan support
  • Grammar
  • Function calling

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don’t know how to start. I would need guidance.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (9 by maintainers)

Most upvoted comments

@bitterspeed It eventually took me longer than I initially anticipated due to breaking changes in llama.cpp that I had to thoroughly test my adaptations for before releasing a new version, but it’s finally released. I’ve tested the new version with your reproduction repo and it seems to work well on my test machine, but let me know whether it actually solved the issue for you.

Thanks for making the reproduction repo.

I was surprised to discover that Electron Forage doesn’t provide any built-in solution to packaging external modules in the asar file under a node_modules directory, which is problematic since copying a single module from the source node_modules directory to the packaged one doesn’t guarantee that its dependencies are also being copied. electron-builder takes care of this automatically and even rebuilds the dependency tree by itself to ensure no unused modules are being copied, so I recommend switching to it if you can.

The pattern I noticed is commonly used to workaround this issue with Electron Forage is to create an src directory with its own package.json that only has dependencies that you want to be unpacked in the final build’s asar file and doesn’t have any devDependencies, and the root package.json has all the rest of the modules you use (both devDependencies and dependencies that are packaged in the final build). In the build process the entire node_modules directory of the src folder is being packed in the final asar file.

I’ve investigated the build generated by repo you provided and found that the code in node-llama-cpp that checks for the prebuilt binaries’ compatibility with the current machine was behaving unexpectedly under this Electron Forage build configuration for some reason. I’ll release a new version of node-llama-cpp with a fix in the next few days that should hopefully fix this issue for you.

What I’m not sure about is why you haven’t encountered this issue on version 3.0.0-beta.44, since this part of the code has been there since that version. I think it may be related somehow to the VitePlugin, since I saw that they decided to retroactively mark it as experimental.