AutoGPT: openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens

⚠️ Search for existing issues first ⚠️

I have searched the existing issues, and there is no existing issue for my problem

Which Operating System are you using?

Windows

Which version of Auto-GPT are you using?

Latest Release

GPT-3.5

Steps to reproduce 🕹

Building a business plan give me this error: openai.error.InvalidRequestError: This model’s maximum context length is 4097 tokens. However, your messages resulted in 10134 tokens. Please reduce the length of the messages.

Current behavior 😯

CRITICISM: We need to ensure that our Python script is optimized for performance and that it is based on accurate and reliable information. We also need to be mindful of potential legal and regulatory challenges, and ensure that our technology infrastructure is secure and reliable. NEXT ACTION: COMMAND = write_to_file ARGUMENTS = {‘filename’: ‘C:\Auto-GPT-0.3.0\autogpt\auto_gpt_workspace\newproject.py’, ‘text’: ‘def generate_improved_code(suggestions, code):\n # TODO: Add code here\n\n return improved_code\n’} SYSTEM: Command write_to_file returned: File written to successfully. Traceback (most recent call last): File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py”, line 196, in _run_module_as_main return _run_code(code, main_globals, None, File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\runpy.py”, line 86, in run_code exec(code, run_globals) File "C:\Auto-GPT-0.3.0\autogpt_main.py", line 5, in <module> autogpt.cli.main() File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py”, line 1130, in call return self.main(*args, **kwargs) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py”, line 1055, in main rv = self.invoke(ctx) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py”, line 1635, in invoke rv = super().invoke(ctx) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py”, line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py”, line 760, in invoke return __callback(*args, **kwargs) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\click\decorators.py”, line 26, in new_func return f(get_current_context(), *args, **kwargs) File “C:\Auto-GPT-0.3.0\autogpt\cli.py”, line 90, in main run_auto_gpt( File “C:\Auto-GPT-0.3.0\autogpt\main.py”, line 171, in run_auto_gpt agent.start_interaction_loop() File “C:\Auto-GPT-0.3.0\autogpt\agent\agent.py”, line 94, in start_interaction_loop assistant_reply = chat_with_ai( File “C:\Auto-GPT-0.3.0\autogpt\llm\chat.py”, line 166, in chat_with_ai agent.summary_memory = update_running_summary( File “C:\Auto-GPT-0.3.0\autogpt\memory_management\summary_memory.py”, line 114, in update_running_summary current_memory = create_chat_completion(messages, cfg.fast_llm_model) File “C:\Auto-GPT-0.3.0\autogpt\llm\llm_utils.py”, line 166, in create_chat_completion response = api_manager.create_chat_completion( File “C:\Auto-GPT-0.3.0\autogpt\llm\api_manager.py”, line 55, in create_chat_completion response = openai.ChatCompletion.create( File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_resources\chat_completion.py”, line 25, in create return super().create(*args, **kwargs) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_resources\abstract\engine_api_resource.py”, line 153, in create response, _, api_key = requestor.request( File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_requestor.py”, line 226, in request resp, got_stream = self._interpret_response(result, stream) File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_requestor.py”, line 619, in _interpret_response self._interpret_response_line( File “C:\Users\xxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\openai\api_requestor.py”, line 682, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model’s maximum context length is 4097 tokens. However, your messages resulted in 10134 tokens. Please reduce the length of the messages. Press any key to continue . . .

Expected behavior 🤔

openai.error.InvalidRequestError: This model’s maximum context length is 4097 tokens. However, your messages resulted in 10134 tokens. Please reduce the length of the messages.

I have followed this instructions and still havent worked: https://github.com/Significant-Gravitas/Auto-GPT/discussions/1061

Your prompt 📝

# Paste your prompt here

Your Logs 📒

<insert your logs here>

About this issue

Original URL
State: closed
Created a year ago
Reactions: 3
Comments: 45 (4 by maintainers)

Most upvoted comments

the issue is in summary_memory.py -> update_running_summary:

when adding “Latest Development:” to the prompt for the history, after running auto-gpt for a while there are more and more entires… need to limit the history which is kept, otherwise prompt gets too long

(that’s at least the issue each time i run into this error)

anyone wants to have a go at a patch? (havent contributed any fixes yet, but done a few locally for testing, could have a go at it)

daice77 on May 17, 2023

Try changing api_manager.py to include a method for chat completion

import openai
from openai.error import InvalidRequestError

# ...

class ApiManager(metaclass=Singleton):
    # ...

    def create_chat_completion(
        self,
        messages: list,  # type: ignore
        model: str | None = None,
        temperature: float = None,
        max_tokens: int | None = None,
        deployment_id=None,
    ) -> str:
        cfg = Config()
        if temperature is None:
            temperature = cfg.temperature
        try:
            if deployment_id is not None:
                response = openai.ChatCompletion.create(
                    deployment_id=deployment_id,
                    model=model,
                    messages=messages,
                    temperature=temperature,
                    max_tokens=max_tokens,
                    api_key=cfg.openai_api_key,
                )
            else:
                response = openai.ChatCompletion.create(
                    model=model,
                    messages=messages,
                    temperature=temperature,
                    max_tokens=max_tokens,
                    api_key=cfg.openai_api_key,
                )
        except InvalidRequestError as e:
            # Handle the specific error, e.g., if it's a token limit error
            if e.code == "invalid_request" and "token limit" in e.message.lower():
                # Implement your logic to reduce the length of messages or split them into smaller parts
                # Modify messages or take appropriate action
                pass
            else:
                # Handle other specific error cases or fallback to a general error handling strategy
                pass
        else:
            if not hasattr(response, "error"):
                logger.debug(f"Response: {response}")
                prompt_tokens = response.usage.prompt_tokens
                completion_tokens = response.usage.completion_tokens
                self.update_cost(prompt_tokens, completion_tokens, model)
        return response

Lets see if something like this would work.

GoZippy on May 19, 2023

Also getting this, using VS Code + Containers

ZylerKade on May 13, 2023

Not sure if there is a solution or workaround to this…my program exits whenever it hits this limit midway…

Rykimaruh on May 8, 2023

#4652 is already merged. Please sync and give it a try.

kinance on Jun 12, 2023

FYI my solution is about ready to test. Anyone want to try it out message me… quite different approach than these others out there…

GoZippy on Jun 12, 2023

so there hasn’t been any news regarding this issue yet?

Rykimaruh on May 16, 2023