node-llama-cpp: NoBinaryFoundError for Windows in Electron when Upgrading from 3.0.0-beta44 -> 3.2.0
Issue description
NoBinaryFoundError for Windows when Upgrading from 3.0.0-beta44 -> 3.2.0
Expected Behavior
I’d expect the WIndows x64 prebuilt binary to be available for use so users do not have to build locally.
Actual Behavior
A NoBinaryFoundError is thrown when trying to start getLlama
Steps to reproduce
- Use Electron + node-llama-cpp
- build on windows
- use getLlama()
Btw, prebuilt binary was found and worked great in 3.0.0-beta44
My Environment
| Dependency | Version |
|---|---|
| Operating System | Win 10 |
| CPU | Intel I7 |
node-llama-cpp version |
3.2.0 |
Additional Context
No response
Relevant Features Used
- Metal support
- CUDA support
- Vulkan support
- Grammar
- Function calling
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, but I don’t know how to start. I would need guidance.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (9 by maintainers)
@bitterspeed It eventually took me longer than I initially anticipated due to breaking changes in
llama.cppthat I had to thoroughly test my adaptations for before releasing a new version, but it’s finally released. I’ve tested the new version with your reproduction repo and it seems to work well on my test machine, but let me know whether it actually solved the issue for you.Thanks for making the reproduction repo.
I was surprised to discover that Electron Forage doesn’t provide any built-in solution to packaging external modules in the asar file under a
node_modulesdirectory, which is problematic since copying a single module from the sourcenode_modulesdirectory to the packaged one doesn’t guarantee that its dependencies are also being copied.electron-buildertakes care of this automatically and even rebuilds the dependency tree by itself to ensure no unused modules are being copied, so I recommend switching to it if you can.The pattern I noticed is commonly used to workaround this issue with Electron Forage is to create an
srcdirectory with its ownpackage.jsonthat only hasdependenciesthat you want to be unpacked in the final build’s asar file and doesn’t have anydevDependencies, and the rootpackage.jsonhas all the rest of the modules you use (bothdevDependenciesanddependenciesthat are packaged in the final build). In the build process the entirenode_modulesdirectory of thesrcfolder is being packed in the final asar file.I’ve investigated the build generated by repo you provided and found that the code in
node-llama-cppthat checks for the prebuilt binaries’ compatibility with the current machine was behaving unexpectedly under this Electron Forage build configuration for some reason. I’ll release a new version ofnode-llama-cppwith a fix in the next few days that should hopefully fix this issue for you.What I’m not sure about is why you haven’t encountered this issue on version
3.0.0-beta.44, since this part of the code has been there since that version. I think it may be related somehow to theVitePlugin, since I saw that they decided to retroactively mark it as experimental.