langchain: Chat agent does not parse properly when model returns a code block

When you ask GPT to produce code, you get an error since the way responses are parsed splits the cleaned output by triple backticks, which doesn’t account for possible usage of triple backticks in the output itself.

To reproduce:

llm = ChatOpenAI(model_name="gpt-3.5-turbo")
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

tools = load_tools(["llm-math"], llm=llm)
agent = initialize_agent(tools, 
                         llm, 
                         agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION, 
                         verbose=True, 
                         memory=memory)


agent.run(input='Write a program for Hello World in Python.')

Traceback:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File [~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py:106](https://vscode-interactive+.vscode-resource.vscode-cdn.net//~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py:106), in ConversationalChatAgent._extract_tool_and_input(self, llm_output)
    [105](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=104) try:
--> [106](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=105)     response = self.output_parser.parse(llm_output)
    [107](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=106)     return response["action"], response["action_input"]

File [~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py:43](https://vscode-interactive+.vscode-resource.vscode-cdn.net//~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py:43), in AgentOutputParser.parse(self, text)
     [42](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=41) if "```" in cleaned_output:
---> [43](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=42)     cleaned_output, _ = cleaned_output.split("```")
     [44](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/agents/conversational_chat/base.py?line=43) if cleaned_output.startswith("```json"):

ValueError: too many values to unpack (expected 2)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
[/Users/sachit/Documents/Documents](https://vscode-interactive+.vscode-resource.vscode-cdn.net/Users/sachit/Documents/Documents) - Sachit’s MacBook Pro[/Projects/ai_assistant/import](https://vscode-interactive+.vscode-resource.vscode-cdn.net/Projects/ai_assistant/import) openai.py in line 1
----> [38](file:///Users/sachit/Documents/Documents%20-%20Sachit%E2%80%99s%20MacBook%20Pro/Projects/ai_assistant/import%20openai.py?line=37) agent.run(input=query)

File [~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py:216](https://vscode-interactive+.vscode-resource.vscode-cdn.net//~/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py:216), in Chain.run(self, *args, **kwargs)
    [213](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py?line=212)     return self(args[0])[self.output_keys[0]]
    [215](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py?line=214) if kwargs and not args:
--> [216](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py?line=215)     return self(kwargs)[self.output_keys[0]]
    [218](file:///Users/sachit/opt/miniconda3/envs/misc/lib/python3.10/site-packages/langchain/chains/base.py?line=217) raise ValueError(
...
{
    "action": "Final Answer",
    "action_input": "Here is an example program for 'Hello World' in Python: \n\n```python\nprint('Hello, World!')\n```"
}
```