micromark: TokenizeContext.sliceSerialize throws in sliceChunks if first chunk of token is Code instead of string

Initial checklist

Affected packages and versions

micromark@3.1.0

Link to runnable example

No response

Steps to reproduce

user@HOST micromark-issue % npm ls micromark
micromark-issue@ /Users/user/micromark-issue
└── micromark@3.1.0

user@HOST micromark-issue % cat issue.mjs 
import { parse } from "micromark/lib/parse";
import { postprocess } from "micromark/lib/postprocess";
import { preprocess } from "micromark/lib/preprocess";

function repro(markdown) {
  console.log("trying...");
  const encoding = undefined;
  const end = true;
  const options = undefined;
  const chunks = preprocess()(markdown, encoding, end);
  const parseContext = parse(options).document().write(chunks);
  const events = postprocess(parseContext);
  for (const event of events) {
    const [ \_, token, context ] = event;
    context.sliceSerialize(token);
  }
  console.log("ok");
}

repro("Heading\\n=======");
repro("\\nHeading\\n=======");
user@HOST micromark-issue % node issue.mjs 
trying...
ok
trying...
file:///Users/user/micromark-issue/node\_modules/micromark/lib/create-tokenizer.js:520
      view[0] = view[0].slice(startBufferIndex)
                        ^

TypeError: view[0].slice is not a function
    at sliceChunks (file:///Users/user/micromark-issue/node\_modules/micromark/lib/create-tokenizer.js:520:25)
    at sliceStream (file:///Users/user/micromark-issue/node\_modules/micromark/lib/create-tokenizer.js:154:12)
    at Object.sliceSerialize (file:///Users/user/micromark-issue/node\_modules/micromark/lib/create-tokenizer.js:149:28)
    at repro (file:///Users/user/micromark-issue/issue.mjs:15:13)
    at file:///Users/user/micromark-issue/issue.mjs:21:1
    at ModuleJob.run (node:internal/modules/esm/module\_job:198:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:385:24)
    at async loadESM (node:internal/process/esm\_loader:88:5)
    at async handleMainPromise (node:internal/modules/run\_main:61:12)
user@HOST micromark-issue % 

Expected behavior

sliceSerialize should always be safe to call in a manner like the above and should return a meaningful string. The presence of a leading \n in Markdown (for example) should not need to be guarded against by library users.

Actual behavior

Exception, see above

Runtime

Node v16

Package manager

npm v7

OS

macOS

Build and bundle tools

Other (please specify in steps to reproduce)

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 20 (19 by maintainers)

Commits related to this issue

Most upvoted comments

Absolutely, thanks! I’ll update my branch and send a proper PR in a few days.