langchain: HNSWLib-node not found when using in a AWS Lambda function

I recently made a simple Typescript function to create a VectorStore using HNSWLib-node. It saves the vector store in a folder and then, in another script file, I load and execute a RetrievalQAChain using OpenAI.

Everything was working fine until I decided to put that in a AWS Lambda Function.

My package.json has the following dependencies:

    "hnswlib-node": "^1.4.2",
    "langchain": "^0.0.59",

Also, I double checked and the hnswlib-node folder is inside “node_modules” folder in my lambda function folder.

However, I keep getting the following error (from CloudWatch Logs):

ERROR	Invoke Error 	{"errorType":"Error","errorMessage":"Please install hnswlib-node as a dependency with,
 e.g. `npm install -S hnswlib-node`",
    "stack":["Error: Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`","    
    at Function.imports (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:161:19)","
    at async Function.getHierarchicalNSW (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:38:37)","
    at async Function.load (/var/task/node_modules/langchain/dist/vectorstores/hnswlib.cjs:123:23)"," 
    at async AMCompanion (/var/task/index.js:18:29)","    at async Runtime.exports.handler (/var/task/index.js:39:22)"]}

Also, this error is not thrown on importing HNSWLib, but only in the following line of code:

const vectorStore = await HNSWLib.load("data", new OpenAIEmbeddings(
            {
                openAIApiKey: process.env.OPENAI_API_KEY,
            } 
        ))

This is my import:

const { HNSWLib } = require("langchain/vectorstores/hnswlib")

It seems I’m not the only one with this problem. See this post

Expeted behavior: code would be executed properly, just like when executed on my local machine.

Actual behavior: the error pasted above.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 15 (1 by maintainers)

Most upvoted comments

+1 also getting this error in a nextjs project

Hi, I found the same problem using this proyect as dependency on a Node 18.14.2 environment running into a RedHat docker container on Azure AKS. On app node_modules folder I can find the hnswlib-node folder but if I look at “node_modules/langchain/node_modules” I only see a “binary-extensions” folder. I think that it’s a problem with dependency management below node 19

@raselSoftNode @joaotextor the solution to this is to upgrade your node version runtime. anything below 18 doesn’t work.