MemGPT: Frequent errors with webgui llm answers when json decoding fails.

I found that most longer answers from the webgui backend will fail to JSON decode. From the data I get to see, it looks like as if the LLM returned an incomplete message. I wonder if one could make the LLM “continue” the message when decoding of it fails. Maybe also try to repair a JSON answer if it is “just missing” the proper ending? I also wonder if one could give the backend a grammar or maximum new token length or how to set any parameters. Is that done inside webgui in that case?

When using “–model dolphin-2.1-mistral-7b” with the same model loaded into webgui I also get this kind of malformed JSON when trying to store into memory > Enter your message: my name is Horst and I am 42 years old. save that in your memory.

Exception: Failed to decode JSON from LLM output:
{
  "function": "core_memory_append",
  "params": {
    "name": "human",
    "content": "Horst, 42 years old, from Germany."
  }
}
{
  "function": "send_message",
  "params": {
    "message": "Got it! Your age and nationality are now saved in my memory."
  }
}