langchain: AzureOpenAI InvalidRequestError: Too many inputs. The max number of inputs is 1.

I might have mitigated the issue by adding the chunk size to the embeddings: embedding = OpenAIEmbeddings(deployment="embeddings",model="text-embedding-ada-002", chunk_size = 1)

In the javscript version of langchain the parameter chunk_size is named batchSize

@huislaw here is my solution, use chunkify function to customize max number of inputs (max=16)

from langchain.document_loaders import WebBaseLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.chat_models import AzureChatOpenAI
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
import logging
from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain.chains import RetrievalQA
import os
from typing import Iterable

logging.basicConfig()
logging.getLogger("langchain.retrievers.multi_query").setLevel(logging.INFO)


def chunkify(arr: Iterable, size: int = 8):
    for i in range(0, len(arr), size):
        yield arr[i : i + size]


embedder = OpenAIEmbeddings(
    openai_api_key=os.getenv("OPENAI_EMBEDDING_API_KEY"),
    openai_api_base=os.getenv("OPENAI_EMBEDDING_API_BASE"),
    openai_api_version=os.getenv("OPENAI_EMBEDDING_API_VERSION"),
    openai_api_type=os.getenv("OPENAI_EMBEDDING_API_TYPE"),
    deployment=os.getenv("OPENAI_EMBEDDING_API_MODEL"),
)


chatllm = AzureChatOpenAI(
    openai_api_key=os.getenv("OPENAI_CHAT_API_KEY"),
    openai_api_base=os.getenv("OPENAI_CHAT_API_BASE"),
    openai_api_version=os.getenv("OPENAI_CHAT_API_VERSION"),
    openai_api_type=os.getenv("OPENAI_CHAT_API_TYPE"),
    deployment_name=os.getenv("OPENAI_CHAT_API_MODEL"),
    temperature=0,
)

with open("document_urls.txt", "r") as F:
    urls = F.read().split("\n")


loader = WebBaseLoader(web_path=urls)
data = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=200, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

vectorstore = Chroma(embedding_function=embedder)
for chunk in chunkify(all_splits):
    vectorstore.add_documents(chunk)

retriever_from_llm = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(), llm=chatllm
)

qa_chain = RetrievalQA.from_chain_type(chatllm, retriever=retriever_from_llm)
result = qa_chain({"query": "How many versions are there in AAVE"})
print(result)

I was able to pass in 16 documents at a time too without the max number of input error, however I had quite a few documents. I used a for loop which worked for me however I had to add time.sleep(2) otherwise I got a rate limit warning:

openai.error.RateLimitError: Requests to the Get a vector representation of a given input that can be easily consumed by machine learning models and algorithms. Operation under Azure OpenAI API version 2023-03-15

see this thread.

example code:

batch = 16
total_docs = len(all_docs)
vectordb = Chroma(persist_directory=persist_directory, embedding_function=embeddings_model)

for i in range(0, total_docs, batch):
    sample_docs = mapping_docs[i:i + batch]
    vectordb.add_documents(sample_docs)
    time.sleep(2) # embarrassing but works
vectordb.persist()

It looks like the team increased the limit, a chunk_size of 16 works for me (deployed text-embedding-ada-002).

I’ve deployed my instance of Azure OpenAI to eastus (maybe quotas differ per Azure Region)

chunk_size here in the Azure OpenAIEmbeddings() is referring to the number of embeddings it creates in parallel as opposed to the langchain chunk_size which is calculating the size of the chunk. chunk_size = 16 worked at this time.

As of 8/5/23, the easiest fix is to pass in chunk_size=16 when creating OpenAIEmbeddings for an Azure deployment. Some of the other solutions here are more complicated than using this built-in functionality. As some have noted, the limit has been increased to 16 from 1.

Confusingly, this value is distinct from the chunk size for text splitting. Here, the configuration tells the OpenAIEmbeddings object to create 16 embeddings at a time, which conforms to the Azure limit. In the TypeScript version of langchain, the name of this configuration is is batchSize.

Not sure if @sunyq1995 means this, but this worked for me, and I think it was faster than doing from_texts

embeddings = OpenAIEmbeddings(
    deployment=embedding_deployment_id,
    model=embedding_model_name,
    chunk_size=1,
    max_retries=10,
    show_progress_bar=True,
)

loader = DataFrameLoader(
    data_df,
    page_content_column="text",
)
text_splitter = TokenTextSplitter(chunk_size=2_000, chunk_overlap=5)
documents = text_splitter.split_documents(loader.load())

returned_embeddings = embeddings.embed_documents(
    [doc.page_content for doc in documents],
)

docsearch = FAISS.from_embeddings(
    text_embeddings=[
        (doc.page_content, embedding)
        for doc, embedding in zip(documents, returned_embeddings)
    ], 
    embedding=embeddings,
    metadatas=[
        doc.metadata
        for doc in documents
    ],
)

How does following code work?

pdf = PyPDFLoader(url)
documents = pdf.load()

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(documents)

for text in texts:
    vectordb.add_documents([text])
vectordb.persist()