langchainjs: Could not import faiss-node (NextJS) when loading from python
Hi, I know this is a duplicate of:
https://github.com/langchain-ai/langchainjs/issues/2107
However, in that issue it says that this has been resolved. While I’m on the latest release: 0.1.2
Which should have the fix for this available here: https://github.com/langchain-ai/langchainjs/pull/2178
My issue is the following: https://github.com/langchain-ai/langchainjs/issues/2107#issuecomment-1891106038
For reference, I’ll post it below:
Hi there,
I’m getting the same type of error message in my console when I try to:
const vectorstore = await FaissStore.loadFromPython("./knowledge", new OpenAIEmbeddings({
modelName: "text-embedding-ada-002"
}));
knowledge is a folder in the root folder that includes 2 files:
index.pkl
index.faiss
I am using the following to see whether my API route in nextjs can reach the files:
const knowledge = path.resolve('./knowledge');
const filenames = fs.readdirSync(knowledge);
console.log(filenames);
this logs the actual filenames, so I know they’re there…
I’m getting the following error message in my console:
Error: Could not import faiss-node. Please install faiss-node as a dependency with, e.g. `npm install -S faiss-node`.
Error: Cannot read properties of undefined (reading 'indexOf')
at FaissStore.importFaiss (webpack-internal:///(rsc)/./node_modules/@langchain/community/dist/vectorstores/faiss.js:363:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async readIndex (webpack-internal:///(rsc)/./node_modules/@langchain/community/dist/vectorstores/faiss.js:292:37)
at async Promise.all (index 1)
at async FaissStore.loadFromPython (webpack-internal:///(rsc)/./node_modules/@langchain/community/dist/vectorstores/faiss.js:295:32)
at async POST (webpack-internal:///(rsc)/./app/api/chat/route.ts:56:29)
at async D:\Development\Persoonlijk\gpt-creator\node_modules\next\dist\compiled\next-server\app-route.runtime.dev.js:6:63251
I’m running this on windows with npm run dev
I’ve already tried installing the package, but I have no clue why my nextjs isn’t importing the files…
My node version is 20.10.0
Is there anything I can do to get this issue sorted?
It says read properties of undefined (reading ‘indexOf’) This to me indicates it can not read the ./knowledge folder with files for some reason. But I haven’t been able to figure out why yet. I could be totally wrong here though.
About this issue
- Original URL
- State: closed
- Created 6 months ago
- Reactions: 1
- Comments: 23 (9 by maintainers)
Commits related to this issue
- fix #4027 — committed to ewfian/langchainjs by ewfian 5 months ago
- fix #4027 loadFromPython failed (#4036) * fix #4027 * update fixture * update int test — committed to langchain-ai/langchainjs by ewfian 5 months ago
- fix #4027 loadFromPython failed (#4036) * fix #4027 * update fixture * update int test — committed to mieslep/langchainjs by ewfian 5 months ago
When your PR has been merged (for the actual fix), I’ll close this github issue as solved.
I’ll do that, thank you for everything so far! I’m really happy with the progress that was made.
EDIT: I created a discussion here: https://github.com/langchain-ai/langchainjs/discussions/4037
@Simbaclaws I just created a Next.js project and reproduced your issue using the
App Routerpattern. From the error, I noticed that faiss-node is being bundled by webpack, which is incorrect since faiss-node is a native module.I found a configuration in the documentation that exempts
faiss-nodefrom bundling. https://nextjs.org/docs/app/api-reference/next-config-js/serverComponentsExternalPackagesAfter adding this config for faiss-node, it should work correctly. Please give it a try.
CC @ewfian could be a Windows issue?