langchain: HNSWLib-node not found when using in a AWS Lambda function
I recently made a simple Typescript function to create a VectorStore using HNSWLib-node. It saves the vector store in a folder and then, in another script file, I load and execute a RetrievalQAChain using OpenAI.
Everything was working fine until I decided to put that in a AWS Lambda Function.
My package.json has the following dependencies:
"hnswlib-node": "^1.4.2",
"langchain": "^0.0.59",
Also, I double checked and the hnswlib-node folder is inside “node_modules” folder in my lambda function folder.
However, I keep getting the following error (from CloudWatch Logs):
ERROR Invoke Error {"errorType":"Error","errorMessage":"Please install hnswlib-node as a dependency with,
e.g. `npm install -S hnswlib-node`",
"stack":["Error: Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`","
at Function.imports (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:161:19)","
at async Function.getHierarchicalNSW (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:38:37)","
at async Function.load (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:123:23)","
at async AMCompanion (/var/task/index.js:18:29)"," at async Runtime.exports.handler (/var/task/index.js:39:22)"]}
Also, this error is not thrown on importing HNSWLib, but only in the following line of code:
const vectorStore = await HNSWLib.load("data", new OpenAIEmbeddings(
{
openAIApiKey: process.env.OPENAI_API_KEY,
}
))
This is my import:
const { HNSWLib } = require("langchain/vectorstores/hnswlib")
It seems I’m not the only one with this problem. See this post
Expeted behavior: code would be executed properly, just like when executed on my local machine.
Actual behavior: the error pasted above.
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (1 by maintainers)
+1 also getting this error in a nextjs project
Hi, I found the same problem using this proyect as dependency on a Node 18.14.2 environment running into a RedHat docker container on Azure AKS. On app node_modules folder I can find the hnswlib-node folder but if I look at “node_modules/langchain/node_modules” I only see a “binary-extensions” folder. I think that it’s a problem with dependency management below node 19
@raselSoftNode @joaotextor the solution to this is to upgrade your node version runtime. anything below 18 doesn’t work.