anthropic-sdk-typescript: "Unexpected end of JSON input" when streaming on edge environments (Vercel Edge, Cloudflare Workers)

This fix was released in v0.17.0!

OK I fixed it, @nyacg was on the right track, the issue is in the LineDecoder class in streaming.ts.

Specifically, it’s because the decode() method is directly ported from this python implementation, but missed an important behavior difference in js’s split() method vs python’s splitlines() method.

The error happens when decode() receives the any inputs that ends in a new line, e.g.: event: content_block_delta\r\n OR \r\n

In both of these cases, js’s split() method will add an extra empty string to the end of the lines array, whereas python’s splitlines() method will not. This is causing empty lines to be passed through to the next layer of sse decoders, causing this issue.

This is also why the previous patch doesn’t work, it dropped tokens because the extra empty line caused whole data packets to be ignored.

It’s got nothing to do with edge env, I suspect some network config in edge causes SSE packets to be smaller making this issue more noticeable.

For now, you can patch the package really easily:

--- a/streaming.js
+++ b/streaming.js
@@ -266,6 +266,9 @@ class LineDecoder {
         }
         const trailingNewline = LineDecoder.NEWLINE_CHARS.has(text[text.length - 1] || '');
         let lines = text.split(LineDecoder.NEWLINE_REGEXP);
+        if (trailingNewline) {
+            lines.pop();
+        }
         if (lines.length === 1 && !trailingNewline) {
             this.buffer.push(lines[0]);
             return [];

This will account for the different split() behavior and align it with python’s splitlines() behavior.

I already created a patch for my llm-api lib if anyone want the patch files. Commit for patch

Fixed in https://github.com/anthropics/anthropic-sdk-typescript/pull/312 which should be released shortly.

Ahh thank you so much for the detailed investigation and proposed patch @dzhng! We’ll test and port this over to our side ASAP.

Amazing turnaround time on this, thank you both @rattrayalex & @RobertCraigie

Doing some debugging myself it looks like a potential bug in the LineDecoder

Here’s an extract of the logs where an error occurs. I’m printing the output of this.decodeText(chunk); in the LineDecoder.

Note: logs go from bottom to top

ata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"ky"}}
event: content_block_delta d
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" with"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" its"}} event: content_block_delta data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" smo"}}
ERROR in iterMessages, sse:  '{"event":"content_block_delta","data":"","raw":["event: content_block_delta"]}' 
event: content_block_delta
ata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":","}}
event: content_block_delta d
ata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"uda"}}
event: content_block_delta d
ata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"\n\nGo"}}

We get the output Gouda, its smoky i.e. the " with" delta is dropped

Generally the chunks that the LineDecoder.decode gets fed are either

1. event: content_block_delta d then
2. ata: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"<the delta>"}}

However an error occurs when the first chunk is just event: content_block_delta (possibly with a trailing space). This leads to an SSE with a sse.data of an empty string which would usually throw an error. If we just continue instead of throwing an error we then get the next SSE with the delta after this one