swc: parseFileSync span bug

Describe the bug

same file parse twice, different output

Input code

//test.jsx
const a = 1

import { parseFileSync } from ‘@swc/core’;

const ast = parseFileSync(‘./test.jsx’, { jsx: true, syntax: ‘ecmascript’, comments: true, dynamicImport: true, decorators: true, decoratorsBeforeExport: true, script: true }) const ast1 = parseFileSync(‘./test.jsx’, { jsx: true, syntax: ‘ecmascript’, comments: true, dynamicImport: true, decorators: true, decoratorsBeforeExport: true, script: true })

console.log(JSON.stringify(ast)) console.log(‘====’) console.log(JSON.stringify(ast1))

{"type":"Module","span":{"start":0,"end":11,"ctxt":0},"body":[{"type":"VariableDeclaration","span":{"start":0,"end":11,"ctxt":0},"kind":"const","declare":false,"declarations":[{"type":"VariableDeclarator","span":{"start":6,"end":11,"ctxt":0},"id":{"type":"Identifier","span":{"start":6,"end":7,"ctxt":0},"value":"a","typeAnnotation":null,"optional":false},"init":{"type":"NumericLiteral","span":{"start":10,"end":11,"ctxt":0},"value":1},"definite":false}]}],"interpreter":null}
====
{"type":"Module","span":{"start":12,"end":23,"ctxt":0},"body":[{"type":"VariableDeclaration","span":{"start":12,"end":23,"ctxt":0},"kind":"const","declare":false,"declarations":[{"type":"VariableDeclarator","span":{"start":18,"end":23,"ctxt":0},"id":{"type":"Identifier","span":{"start":18,"end":19,"ctxt":0},"value":"a","typeAnnotation":null,"optional":false},"init":{"type":"NumericLiteral","span":{"start":22,"end":23,"ctxt":0},"value":1},"definite":false}]}],"interpreter":null}

first ast span start at 0, second start at 12

Config

{
    // Please copy and paste your .swcrc file here
}

Expected behavior A clear and concise description of what you expected to happen.

Version The version of @swc/core:

@swc/core”: “^1.2.46”,

Additional context Add any other context about the problem here.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 13
  • Comments: 27 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Here is the location of the issue.

Every time transform_sync is called, SourceMap#new_source_file is called and the start_pos is set.

If you are using transformSync and transforming one file at at time, there seems to be no way to reset the start pos. And spans use the SourceMap start pos.

#[js_function(4)]
pub fn transform_sync(cx: CallContext) -> napi::Result<JsObject> {
    exec_transform(cx, |c, src, options| {
        Ok(c.cm.new_source_file(
            if options.filename.is_empty() {
                FileName::Anon
            } else {
                FileName::Real(options.filename.clone().into())
            },
            src,
        ))
    })
}
    pub fn new_source_file(&self, filename: FileName, src: String) -> Lrc<SourceFile> {
        // The path is used to determine the directory for loading submodules and
        // include files, so it must be before remapping.
        // Note that filename may not be a valid path, eg it may be `<anon>` etc,
        // but this is okay because the directory determined by `path.pop()` will
        // be empty, so the working directory will be used.
        let unmapped_path = filename.clone();

        let (filename, was_remapped) = match filename {
            FileName::Real(filename) => {
                let (filename, was_remapped) = self.path_mapping.map_prefix(filename);
                (FileName::Real(filename), was_remapped)
            }
            other => (other, false),
        };

        // We hold lock at here to prevent panic
        // If we don't do this, lookup_char_pos and its family **may** panic.
        let mut files = self.files.borrow_mut();

        let start_pos = self.next_start_pos(src.len());

        let source_file = Lrc::new(SourceFile::new(
            filename,
            was_remapped,
            unmapped_path,
            src,
            Pos::from_usize(start_pos),
        ));

Can we get a config option to not set the start_pos from the end of the previous one? It should probably be the default for the public js api.


NOTE: If let start_pos = self.next_start_pos(src.len()); is set to zero on each compile, source maps actually break…the originalLine in the source map is always 1, and the filename is something like jsx-config-pragmaFrag.js.

We’ve also at first had problems with using spans (in node/typescript). Apart from taking into account the offset as mentioned in other comments above its also important to keep in mind that the span’s start/end refer to byte positions and not char positions. So this was helpful in our context to get the right substrings from SWC’s spans:

export class StringAsBytes {
    private string: Uint8Array;
    private decoder: TextDecoder;

    constructor(string: string) {
        this.decoder = new TextDecoder();
        this.string = (new TextEncoder()).encode(string);
    }

    /**
     * Returns a slice of the string by providing byte indices.
     * @param from - Byte index to slice from
     * @param to - Optional byte index to slice to
     */
    public slice(from: number, to?: number): string {
        return this.decoder.decode(
            new DataView(this.string.buffer, from, to !== undefined ? to - from : undefined)
        );
    }
}

Hope this is helpful.

By design,just change the file name passed into the parseFileSync you’ll get the same result.

It’s not a bug. If you want span to start with 0, you can create new instance of Compiler, which have same apis.

By design,just change the file name passed into the parseFileSync you’ll get the same result.

I think it’s a bug

This bug is kind of deal breaker for switching to swc, In our product we need to invoke parser multiple times in the same process. Increasing span.start with each invocation will make swc stopped working at some point – especially when parsing huge file.