LlamaIndexTS: Error: Unknown node type: DOCUMENT

@himself65 bumped to 0.5.24 to see If the https://github.com/run-llama/LlamaIndexTS/issues/1176 is fixed. It is but now I’m getting the following error:

Error: Unknown node type: DOCUMENT at splitNodesByType (webpack-internal:///(rsc)/…/…/node_modules/@llamaindex/core/dist/schema/index.js:708:19) at createMessageContent (webpack-internal:///(rsc)/…/…/node_modules/llamaindex/dist/synthesizers/utils.js:11:94) at DefaultContextGenerator.generate (webpack-internal:///(rsc)/…/…/node_modules/llamaindex/dist/engines/chat/DefaultContextGenerator.js:50:107) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async ContextChatEngine.prepareRequestMessages (webpack-internal:///(rsc)/…/…/node_modules/llamaindex/dist/engines/chat/ContextChatEngine.js:470:25) at async ContextChatEngine.chat (webpack-internal:///(rsc)/…/…/node_modules/llamaindex/dist/engines/chat/ContextChatEngine.js:437:33) at async Object.start (webpack-internal:///(rsc)/./src/app/api/chat/route.ts:60:32)

Note: This error was not present in the v0.5.20. Something that I initially described in https://github.com/run-llama/LlamaIndexTS/issues/1172

I can repro the issue exactly in the same way as I’ve mentioned in the ticket above. From the ticket:

I’m using llamaindex at app/api/chat/route.ts , in other words, I’m using Route Handlers When I start my app and open /home next.js compiles the app and I can call llamaindex at app/api/chat/route.ts without any error. Then, when I navigate to another page (e.g, /products) next.js does another compilation. After this when I call the handler at app/api/chat/route.ts I get the error above.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (14 by maintainers)

Most upvoted comments

@himself65 after googling around and checking the issue that you’ve linked in https://github.com/run-llama/LlamaIndexTS/pull/1214

What’s the chance of some change after 0.5.20 (last one that was working without complaining) caused CJS and ESM bundling as described in this article https://www.codejam.info/2024/02/esm-cjs-dupe.html ?

I try this on my locally. It runs well, I’m guessing you are using different version of llamaindex & @llamaindex/core

@himself65 here are the versions that I’m currently using

pnpm why -F @web llamaindex
Legend: production dependency, optional only, dev only

@web@6.0.1 /home/Dev/web/apps/web

dependencies:
llamaindex 0.6.2
pnpm why -F @web @llamaindex/core
Legend: production dependency, optional only, dev only

@web@6.0.1 /home/Dev/web/apps/web

dependencies:
llamaindex 0.6.2
├─┬ @llamaindex/cloud 0.2.6
│ └── @llamaindex/core 0.2.2 peer
├── @llamaindex/core 0.2.2
├─┬ @llamaindex/groq 0.0.3
│ └─┬ @llamaindex/openai 0.1.4
│   └── @llamaindex/core 0.2.2
└─┬ @llamaindex/openai 0.1.4
  └── @llamaindex/core 0.2.2

Here’s my next.config.mjs

/**
 * @type {import("next").NextConfig}
 */

import { fileURLToPath } from "url";
import _jiti from "jiti";
import withLlamaIndex from "llamaindex/next";

const jiti = _jiti(fileURLToPath(import.meta.url));

// Import env files to validate at build time. Use jiti so we can load .ts files in here.
jiti("./src/env");

const isStaticExport = "false";

const nextConfig = {
  basePath: process.env.NEXT_PUBLIC_BASE_PATH,
  env: {
    BUILD_STATIC_EXPORT: isStaticExport,
  },
  // Trailing slashes must be disabled for Next Auth callback endpoint to work
  // https://stackoverflow.com/a/78348528
  trailingSlash: false,
  modularizeImports: {
    "@mui/icons-material": {
      transform: "@mui/icons-material/{{member}}",
    },
    "@mui/material": {
      transform: "@mui/material/{{member}}",
    },
    "@mui/lab": {
      transform: "@mui/lab/{{member}}",
    },
  },
  webpack(config) {
    config.module.rules.push({
      test: /\.svg$/,
      use: ["@svgr/webpack"],
    });

    // To allow chatbot to work
    // Extracted from: https://github.com/neondatabase/examples/blob/main/ai/llamaindex/rag-nextjs/next.config.mjs
    config.resolve.alias = {
      ...config.resolve.alias,
      sharp$: false,
      "onnxruntime-node$": false,
    };

    return config;
  },
  ...(isStaticExport === "true" && {
    output: "export",
  }),
};

export default withLlamaIndex(nextConfig);

All llamaindex-related code is located in app/api/chat

config.ts

import { OpenAI, OpenAIEmbedding, Settings } from "llamaindex";

import { env } from "~/env";

const aiGlobalConfig = globalThis as unknown as {
  llmModel?: OpenAI;
  embedModel?: OpenAIEmbedding;
};

const llmModel =
  aiGlobalConfig.llmModel ??
  new OpenAI({
    apiKey: env.OPENAI_API_KEY,
    model: env.OPENAI_MODEL_NAME,
  });

if (env.NODE_ENV !== "production") {
  aiGlobalConfig.llmModel = llmModel;
}

const embedModel =
  aiGlobalConfig.embedModel ??
  new OpenAIEmbedding({
    apiKey: env.OPENAI_API_KEY,
    model: env.OPENAI_EMBED_MODEL_NAME,
  });

if (env.NODE_ENV !== "production") {
  aiGlobalConfig.embedModel = embedModel;
}

// LlamaIndex Settings
Settings.llm = llmModel;

Settings.embedModel = embedModel;

export { Settings };

verctor-store.ts

import { PGVectorStore } from "llamaindex/vector-store/PGVectorStore";

import { env } from "~/env";

const DIMMS = env.EMBED_DIM;

const CONN_STRING = env.VECTOR_STORE_PG_URL;

const globalForVectorStore = globalThis as unknown as {
  vectorStore: PGVectorStore | undefined;
};

const storedVectorStore =
  globalForVectorStore.vectorStore ??
  new PGVectorStore({
    dimensions: DIMMS,
    connectionString: CONN_STRING,
  });

if (env.NODE_ENV !== "production") {
  globalForVectorStore.vectorStore = storedVectorStore;
}

const vectorStore = storedVectorStore;

export { vectorStore };

route.ts

import type { NextRequest } from "next/server";
import { ContextChatEngine, serviceContextFromDefaults, VectorStoreIndex } from "llamaindex";

import type { ChatBotPayload } from "./types";
import { env } from "~/env";
import { Settings } from "./config";
import { vectorStore } from "./vector-store";

/**
 * Key used to store the space ID in the metadata.
 */
const METADATA_SPACE_ID_KEY = "space_id";

export async function POST(request: NextRequest) {
  try {
    const { messages = [], spaces: confluenceSpaces = [] } =
      (await request.json()) as ChatBotPayload;

    if (confluenceSpaces.length === 0) {
      throw new Error("No confluence spaces provided.");
    }

    const userMessages = messages.filter((i) => i.role === "user");
    const query = userMessages[userMessages.length - 1]?.content;

    if (!query) {
      throw new Error("No query provided.");
    }

    const serviceContext = serviceContextFromDefaults({ embedModel: Settings.embedModel });
    const index = await VectorStoreIndex.fromVectorStore(vectorStore, serviceContext);
    const retriever = index.asRetriever({
      topK: {
        TEXT: env.TOP_K_SIMILARITY_TEXT,
        IMAGE: env.TOP_K_SIMILARITY_IMAGE,
      },
      filters: {
        // Limit the search to the allowed confluence spaces.
        filters: confluenceSpaces.map((spaceId) => ({
          key: METADATA_SPACE_ID_KEY,
          value: spaceId,
          operator: "==",
        })),
        condition: "or",
      },
    });
    const chatEngine = new ContextChatEngine({
      retriever,
    });

    const encoder = new TextEncoder();
    const customReadable = new ReadableStream({
      async start(controller) {
        const stream = await chatEngine.chat({
          message: query,
          chatHistory: messages,
          stream: true,
          verbose: true,
        });
        for await (const chunk of stream) {
          controller.enqueue(encoder.encode(chunk.response));
        }
        controller.close();
      },
    });
    return new Response(customReadable, {
      headers: {
        Connection: "keep-alive",
        "Content-Encoding": "none",
        "Cache-Control": "no-cache, no-transform",
        "Content-Type": "text/plain; charset=utf-8",
      },
    });
  } catch (error) {
    const errorMessage =
      error instanceof Error ? error.message : "An error ocurred while processing the request.";

    return Response.json(
      { error: errorMessage },
      {
        headers: { "Content-Type": "application/json" },
        status: 500,
      },
    );
  }
}