crewAI: [BUG] system_prompt incompatibility
Description
I use the Mistral model (deployed on local server, access via custom API), which doesn’t support the system_prompt. I’m also using output_pydantic=True in one of my tasks. When the LLM generates output that doesn’t align with the Pydantic model, CrewAI uses an internal prompt to ask the LLM to regenerate the output. This internal prompt includes a system_prompt, and after that, my LLM crashes.
P.S. I’m aware of the use_system_prompt=False flag when initializing an Agent, but it doesn’t resolve the issue.
Steps to Reproduce
-
Initialize an Agent with an LLM that does not support system_prompt.
-
Initialize a Task with output_pydantic=True.
-
Run your crew until the model generates an invalid Pydantic output.
-
It will crash after CrewAI uses a prompt to request a regeneration attempt.
Expected behavior
The model doesn’t crash
Screenshots/Code snippets
from myfile import MyPydanticModel, my_config, my_condition
from crewai.tasks.conditional_task import ConditionalTask
from crewai.project import task
@task
def resolve_requests(self) -> ConditionalTask:
return ConditionalTask(
condition=my_condition,
config=my_config,
verbose=True,
output_pydantic=MyPdanticModel
)
Operating System
Ubuntu 20.04
Python Version
3.10
crewAI Version
0.108.0
crewAI Tools Version
0.38.1
Virtual Environment
Venv
Evidence
Error code: 400 - {‘object’: ‘error’, ‘message’: ‘Conversation roles must alternate user/assistant/…’, ‘type’: ‘BadRequestError’, ‘param’: None, ‘code’: 400}
Possible Solution
When I comment 31-32 rows in /src/crewai/utilities/converter.py:
{“role”: “system”, “content”: self.instructions}, {“role”: “user”, “content”: self.text}
and add this row: {“role”: “user”, “content”: self.instructions + “/n” + self.text}
All works as expected.
Additional context
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (11 by maintainers)
Commits related to this issue
- fix: Respect use_system_prompt in Converter regeneration Fixes #2571 Co-authored-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com> Co-Authored-By: Joe Moura <joao@crewai.com> — committed to crewAIInc/crewAI by devin-ai-integration[bot] 2 years ago
@lucasgomide @Vidit-Ostwal looks good for me, thank you!
@lucasgomide from my side the PR is ready, have added relevant test cases as well. As far as I understand, the
llmcall made, will not be compatible with mistral and have fixed this.If @maxkochanoff, can take a look at the PR, and confirm if these changes will sort the issue. Then I think the PR is good to go.
I just self-assigned to me to keep on my radar. I’ll tackle that as soon as possible. @Vidit-Ostwal i saw you are interacting, feel free to take this one… I probably will take this one next week (will keep you posted)