llama_index: [Bug]: map_httpcore_exceptions while invoking query_engine.query in colab

Bug Description

New to llama_index. Thank you for all the efforts towards developing and maintaining it. I’m trying to reproduce the starter example in colab using my own tabulated data. Works fine until I try to query:

Following works fine -

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.embeddings import resolve_embed_model
from llama_index.llms.ollama import Ollama

documents = SimpleDirectoryReader("data").load_data()

# bge embedding model
Settings.embed_model = resolve_embed_model("local:BAAI/bge-small-en-v1.5")

# ollama
Settings.llm = Ollama(model="mistral", request_timeout=30.0)

index = VectorStoreIndex.from_documents(
    documents,
)
query_engine = index.as_query_engine()

Error is thrown when trying to query: response = query_engine.query("What did the author write about?")

I have shared the error in the logs section.

Version

llama-index-0.10.27

Steps to Reproduce

Trying to reproduce starter example local of https://docs.llamaindex.ai/en/stable/getting_started/starter_example_local/ in colab.

Relevant Logs/Tracbacks

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 233, in handle_request
    resp = self._pool.handle_request(req)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 99, in handle_request
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 76, in handle_request
    stream = self._connect(request)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 122, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_backends/sync.py", line 205, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 99] Cannot assign requested address

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/content/starter.py", line 18, in <module>
    response = query_engine.query("What did the author write about?")
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/base/base_query_engine.py", line 53, in query
    query_result = self._query(str_or_query_bundle)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/query_engine/retriever_query_engine.py", line 190, in _query
    response = self._response_synthesizer.synthesize(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/response_synthesizers/base.py", line 241, in synthesize
    response_str = self.get_response(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/response_synthesizers/compact_and_refine.py", line 43, in get_response
    return super().get_response(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/response_synthesizers/refine.py", line 183, in get_response
    response = self._give_response_single(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/response_synthesizers/refine.py", line 238, in _give_response_single
    program(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/response_synthesizers/refine.py", line 84, in __call__
    answer = self._llm.predict(
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/instrumentation/dispatcher.py", line 211, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/llms/llm.py", line 430, in predict
    chat_response = self.chat(messages)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/core/llms/callbacks.py", line 144, in wrapped_llm_chat
    f_return_val = f(_self, messages, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/llama_index/llms/ollama/base.py", line 101, in chat
    response = client.post(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1145, in post
    return self.request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 827, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1015, in _send_single_request
    response = transport.handle_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 99] Cannot assign requested address

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 3
  • Comments: 26

Most upvoted comments

I was facing the same issue on running ollama and llamaindex in docker, I added the base url in this line to connect to the ollama docker instance, and this error did not appear again. I hope it helps.

llm = Ollama(model="llama3:latest", request_timeout=500.0, base_url=OLLAMA_BASE_URL)

In my case the base_url was http://<container_name>:11434

Experiencing the same issue, working well in a local notebook but not on Google Colab…

Based on the error message you provided, it seems like you’re encountering a network-related issue when trying to perform a query with LlamaIndex in a Google Colab environment. This error typically occurs when an application is trying to bind or connect to an IP address and port that is not available or not allowed. In the context of Google Colab, this could be due to restrictions on outbound connections or the specific way networking is handled within the Colab environment.

The role of network libraries in the querying process of LlamaIndex involves two main components: NetworkQueryEngine and NetworkRetriever. The NetworkQueryEngine is responsible for processing and executing queries against the data managed by LlamaIndex, while the NetworkRetriever is tasked with the retrieval of data from the network. Together, these components facilitate the querying process by interpreting query requests, fetching the necessary data from the network, and ensuring that users or applications receive the requested information in a timely and efficient manner.

During the querying process, LlamaIndex performs specific network operations such as asynchronous querying of contributors, gathering results, response synthesis, and callback management. These operations are encapsulated within the _query and _aquery methods for synchronous and asynchronous querying, respectively.

To resolve this issue, you might want to consider the following steps:

  1. Check Network Permissions: Ensure that Google Colab allows the type of network connection that llama-index is attempting to make. Google Colab has limitations on outbound connections, especially to non-standard ports or addresses.

  2. Update Libraries: The pyproject.toml file specifies llama-index version ^0.10.12, which means it will use the latest version compatible with 0.10.x. If 0.10.27 is causing issues, you might want to explicitly set a version of llama-index that you know works well within Google Colab, if such a version exists.

  3. Use a Proxy or Different Network Configuration: If llama-index is attempting to connect to a service or address that is not allowed in Google Colab, using a proxy or configuring it to use a different address or method of connection (if supported by llama-index) might resolve the issue.

  4. Contact Support or Check Documentation: Check the documentation for llama-index and llama-index-networks for any notes on compatibility with Google Colab or similar environments.

I also found a similar issue in the LlamaIndex repository where the user encountered a problem while querying in a Colab environment. The suggested solution was to correct the document’s argument. Since it is a file, use the input argument input_files to send the document in. Here’s the corrected code:

from llama_index import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader(input_files = ["test2.txt"]).load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
query_engine.query("what is my name?")

You can find more details about this issue here.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Hey @sunilnagpal, great to see you back! Hope you’re doing well despite this pesky connection error giving you a bit of a headache. Let’s see what we can do to sort it out.

Based on the error message you’ve shared, it seems like you’re encountering a network-related issue. This error is typically not directly related to the LlamaIndex code, but rather to the network settings in your Google Colab environment. Here are some general steps you can take to address this issue:

  1. Retry Mechanism: Implement a retry mechanism in your query execution logic. Network issues can be transient, and retrying the request after a short delay may resolve the issue.

  2. Check Resource Limits: Ensure you’re not exceeding any resource limits in Google Colab, such as the number of open connections or outbound requests. Google Colab has limits to prevent abuse, and exceeding these limits might result in network-related errors.

  3. Use a Different Network Library: If you’re using a specific HTTP client library and encountering this issue, consider switching to another library. For example, if you’re using httpx, you might try requests or vice versa, to see if the issue persists.

  4. Review Colab Network Restrictions: Google Colab might have restrictions on certain types of network traffic or destinations. Ensure that your query’s destination is not blocked or restricted by Google Colab’s environment.

  5. Local Execution: As a last resort, if the network issue cannot be resolved within Google Colab, consider running your LlamaIndex query execution code locally or in another environment where you have more control over the network configuration.

Regarding your code snippet, it seems correct. However, there was a similar issue reported in the past where the user was not correctly loading the documents. Make sure you’re correctly loading your documents using the SimpleDirectoryReader class. Here’s the corrected code snippet from the similar issue:

from llama_index import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader(input_files = ["test2.txt"]).load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
query_engine.query("what is my name?")

You can find more details about this issue here.

Lastly, please note that there are known limitations with using LlamaIndex in a Google Colab environment. Google Colab typically uses Python 3.7, and LlamaIndex requires Python version 3.10 specifically. This means LlamaIndex may not be compatible with the default Python version in Google Colab. You can find more details about this limitation in the BUILD file.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.