llama_index: [Bug]: Not able to make call to Ollama multimodal model in the cookbook
Bug Description
When I trying to follow the steps int the cookbook, I was getting an error saying that there’s a ResponseError: calling Ollama. However, I have installed Ollama successfully and was able to make model call via curl command. I suspect that this has something to do with http://localhost:11434.
Version
llama-index 0.9.44
Steps to Reproduce
Following the steps in the cookbook
from pydantic import BaseModel
class Restaurant(BaseModel):
"""Data model for an restaurant."""
restaurant: str
food: str
discount: str
price: str
rating: str
review: str
from llama_index.program import MultiModalLLMCompletionProgram
from llama_index.output_parsers import PydanticOutputParser
prompt_template_str = """\
{query_str}
Return the answer as a Pydantic object. The Pydantic schema is given below:
"""
mm_program = MultiModalLLMCompletionProgram.from_defaults(
output_parser=PydanticOutputParser(Restaurant),
image_documents=image_documents,
prompt_template_str=prompt_template_str,
multi_modal_llm=mm_model,
verbose=True,
)
response = mm_program(query_str="Can you summarize what is in the image?")
for res in response:
print(res)
Relevant Logs/Tracbacks
{ “name”: “ResponseError”, “message”: “”, “stack”: "--------------------------------------------------------------------------- ResponseError Traceback (most recent call last) Cell In[13], line 1 ----> 1 response = mm_program(query_str="Can you summarize what is in the image?") 2 for res in response: 3 print(res)
File /usr/local/lib/python3.11/site-packages/llama_index/program/multi_modal_llm_program.py:85, in MultiModalLLMCompletionProgram.call(self, llm_kwargs, *args, **kwargs) 82 llm_kwargs = llm_kwargs or {} 83 formatted_prompt = self._prompt.format(llm=self._multi_modal_llm, **kwargs) —> 85 response = self._multi_modal_llm.complete( 86 formatted_prompt, 87 image_documents=self._image_documents, 88 **llm_kwargs, 89 ) 91 raw_output = response.text 92 if self._verbose:
File /usr/local/lib/python3.11/site-packages/llama_index/multi_modal_llms/ollama.py:158, in OllamaMultiModal.complete(self, prompt, image_documents, formatted, **kwargs) 155 """Complete.""" 156 import ollama –> 158 response = ollama.generate( 159 model=self.model, 160 prompt=prompt, 161 images=image_documents_to_base64(image_documents), 162 stream=False, 163 options=self._model_kwargs, 164 ) 165 return CompletionResponse( 166 text=response["response"], 167 raw=response, 168 additional_kwargs=get_additional_kwargs(response, ("response",)), 169 )
File /usr/local/lib/python3.11/site-packages/ollama/_client.py:111, in Client.generate(self, model, prompt, system, template, context, stream, raw, format, images, options, keep_alive) 108 if not model: 109 raise RequestError(‘must provide a model’) –> 111 return self._request_stream( 112 ‘POST’, 113 ‘/api/generate’, 114 json={ 115 ‘model’: model, 116 ‘prompt’: prompt, 117 ‘system’: system, 118 ‘template’: template, 119 ‘context’: context or [], 120 ‘stream’: stream, 121 ‘raw’: raw, 122 ‘images’: [_encode_image(image) for image in images or []], 123 ‘format’: format, 124 ‘options’: options or {}, 125 ‘keep_alive’: keep_alive, 126 }, 127 stream=stream, 128 )
File /usr/local/lib/python3.11/site-packages/ollama/_client.py:82, in Client._request_stream(self, stream, *args, **kwargs) 76 def _request_stream( 77 self, 78 *args, 79 stream: bool = False, 80 **kwargs, 81 ) -> Union[Mapping[str, Any], Iterator[Mapping[str, Any]]]: —> 82 return self._stream(*args, **kwargs) if stream else self._request(*args, **kwargs).json()
File /usr/local/lib/python3.11/site-packages/ollama/_client.py:58, in Client._request(self, method, url, **kwargs) 56 response.raise_for_status() 57 except httpx.HTTPStatusError as e: —> 58 raise ResponseError(e.response.text, e.response.status_code) from None 60 return response
ResponseError: " }
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 17 (3 by maintainers)
PR https://github.com/run-llama/llama_index/pull/11253
@logan-markewich I can add it back