twinny: JSON parse errors handling streamed responses

Describe the bug

When I run a local Ollama server on my M1 MacBook, Twinny seems to always receive a single HTTP chunk with a complete JSON object it containing the query response.

However, when running Ollama on a server with a decent GPU, sometimes the responses are large enough that they get split up into multiple chunks, and Twinny attempts to parse incomplete JSON objects.

I believe there is a bug in Twinny’s handling of Ollama’s “chunked” HTTP transfer encoding, because if I inspect the object being passed to JSON.parse in the extension I see incomplete objects, and a stack trace like this from Twinny:

2024-01-23 12:42:54.158 [error] SyntaxError: Expected ',' or ']' after array element in JSON at position 11330
	at JSON.parse (<anonymous>)
	at onData (/Users/nick/.vscode/extensions/rjmacarthy.twinny-2.6.17/out/extension.js:685:46)
	at IncomingMessage.<anonymous> (/Users/nick/.vscode/extensions/rjmacarthy.twinny-2.6.17/out/extension.js:988:17)
	at IncomingMessage.emit (node:events:513:28)
	at Readable.read (node:internal/streams/readable:539:10)
	at flow (node:internal/streams/readable:1023:34)
	at resume_ (node:internal/streams/readable:1004:3)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

It looks like Twinny is receiving a chunk and trying to parse it, but the chunk is not guaranteed to be valid JSON data:

onData: (chunk, onDestroy) => {
              const json = JSON.parse(chunk)
              completion = completion + json.response

I don’t notice this locally, just when running against a server.

To Reproduce Steps to reproduce the behavior:

1. set up Twinny to use an Ollama server that will return large responses (e.g. 15kB)
2. use Twinny in code completion mode

Expected behavior

Twinny should accept the completion suggestion from the server and display it in the editor.

Actual Behaviour

Twinny attempts to parse an incomplete response, resulting in a JSON parsing stack trace, and no suggestion is shown.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: MacOS

Additional context Add any other context about the problem here.