llama_index: [Bug]: Bedrock Integration with Claude3 fails
Bug Description
I am trying to run Claude 3 sonnet model with llama index Bedrock. But the code is not running. Is Claude 3 not available yet on llama index? The code i am trying to run is below:
`from llama_index.llms.bedrock import Bedrock
llm = Bedrock(model=“anthropic.claude-3-sonnet-20240229-v1:0”, client=bedrock_runtime)`
My env: llama-index 0.10.17 llama-index-agent-openai 0.1.5 llama-index-cli 0.1.8 llama-index-core 0.10.17 llama-index-embeddings-openai 0.1.6 llama-index-indices-managed-llama-cloud 0.1.3 llama-index-legacy 0.9.48 llama-index-llms-anthropic 0.1.5 llama-index-llms-bedrock 0.1.3 llama-index-llms-openai 0.1.7 llama-index-multi-modal-llms-openai 0.1.4 llama-index-program-openai 0.1.4 llama-index-question-gen-openai 0.1.3 llama-index-readers-file 0.1.8 llama-index-readers-llama-parse 0.1.3 llama-index-vector-stores-chroma 0.1.5 llama-index-vector-stores-postgres 0.1.2 llama-parse 0.3.7 llamaindex-py-client 0.1.13
Can you please help? Thanks
Version
0.10.17
Steps to Reproduce
`from llama_index.llms.bedrock import Bedrock
llm = Bedrock(model=“anthropic.claude-3-sonnet-20240229-v1:0”, client=bedrock_runtime)`
Relevant Logs/Tracbacks
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 21 (2 by maintainers)
Hi @brycecf @ayseozgun @danielftapiar @Aniket-ByteIQ ,
I am getting the below error. Did any of you get the same error? File “C:\Users\Jay\coding_projects\RAG_AWS_Neptune\venv\Lib\site-packages\llama_index\llms\bedrock\utils.py”, line 157, in get_request_body if len(prompt) > 0 and prompt[0][“role”] == “system”: ~~~~~~~~~^^^^^^^^ TypeError: string indices must be integers, not ‘str’
My code: from llama_index.llms.bedrock import Bedrock from llama_index.core.base.llms.types import ChatMessage
llm = Bedrock(model=“anthropic.claude-3-sonnet-20240229-v1:0”, region_name=“ap-south-1”) messages = [ ChatMessage(role=“system”, content=“You are a pirate with a colorful personality”), ChatMessage(role=“user”, content=“Tell me a story”), ] response = llm.chat(messages=messages)
This should be resolved now. I use Claude Sonnet regularly with LlamaIndex Bedrock.
@ayseozgun can you confirm please so this can be closed?
Same thing here. I have a PR open that will (hopefully) be merged soon to fix this.
Based on the information you’ve provided, it seems like you’re trying to use the Claude 3 sonnet model with the LlamaIndex Bedrock. The model name you’re using, “anthropic.claude-3-sonnet-20240229-v1:0”, might not be correct or available in the LlamaIndex Bedrock.
In the LlamaIndex Bedrock, the Claude 3 sonnet model is referred to as “anthropic.claude-v2:1”. You can find this information in the
CHAT_ONLY_MODELSdictionary in the llama-index-integrations/llms/llama-index-llms-bedrock/llama_index/llms/bedrock/utils.py file.Here’s how you can modify your code:
Also, make sure that your environment is set up to handle the specific requirements of chat-based models. This includes starting prompts with “Human:” and ending with “Assistant:”, as indicated in the comments above the
CHAT_ONLY_MODELSdictionary.If you’re still facing issues, it would be helpful to have more specific details about the errors or problems you’re encountering. This will allow for more targeted assistance.
Sources
About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.