wakaru: [unminify-bugs] Multiple errors during unminify with CLI 0.0.5

Describe the bug

Summary of warnings/errors:

  • ???
    • Multiple exports of “default” found, only the last one will be kept
      • @pionxzh It would be useful if this message mentioned what module was being processed at the time
  • stage2-unpacked/pages/_app/module-85695.js
    • Error running rule un-parameters-lebab on module-85695.js SyntaxError: Unexpected token yyyy
    • Failed to parse rule module-85695.js with jscodeshift in rule un-jsx SyntaxError: Unexpected token, expected “,” (4948:12)
      • @pionxzh Instead of “Failed to parse rule” this probably should be “Failed to parse module” or similar I think?
    • Failed to parse rule module-85695.js with jscodeshift in rule un-iife SyntaxError: Unexpected token, expected “,” (4948:12)
    • Failed to parse rule module-85695.js with jscodeshift in rule un-es6-class SyntaxError: Unexpected token, expected “,” (4948:12)
    • Failed to parse rule module-85695.js with jscodeshift in rule un-async-await SyntaxError: Unexpected token, expected “,” (4948:12)
    • Error running rule prettier on module-85695.js SyntaxError: Unexpected token, expected “,” (4948:13)
  • stage2-unpacked/pages/_app/module-73797.js
    • Error running rule un-parameters-lebab on module-73797.js SyntaxError: Identifier ‘Z’ has already been declared
    • Failed to parse rule module-73797.js with jscodeshift in rule un-jsx SyntaxError: Identifier ‘Z’ has already been declared. (104:13)
    • Failed to parse rule module-73797.js with jscodeshift in rule un-iife SyntaxError: Identifier ‘Z’ has already been declared. (104:13)
    • Failed to parse rule module-73797.js with jscodeshift in rule un-es6-class SyntaxError: Identifier ‘Z’ has already been declared. (104:13)
    • Failed to parse rule module-73797.js with jscodeshift in rule un-async-await SyntaxError: Identifier ‘Z’ has already been declared. (104:13)
  • stage2-unpacked/pages/_app/module-33562.js
    • Error running rule un-parameters-lebab on module-33562.js SyntaxError: Identifier ‘Z’ has already been declared
    • Failed to parse rule module-33562.js with jscodeshift in rule un-jsx SyntaxError: Identifier ‘Z’ has already been declared. (91:13)
    • Failed to parse rule module-33562.js with jscodeshift in rule un-iife SyntaxError: Identifier ‘Z’ has already been declared. (91:13)
    • Failed to parse rule module-33562.js with jscodeshift in rule un-es6-class SyntaxError: Identifier ‘Z’ has already been declared. (91:13)
    • Failed to parse rule module-33562.js with jscodeshift in rule un-async-await SyntaxError: Identifier ‘Z’ has already been declared. (91:13)
  • stage2-unpacked/pages/_app/module-34408.js
    • Error running rule un-parameters-lebab on module-34408.js SyntaxError: Identifier ‘Z’ has already been declared
    • Failed to parse rule module-34408.js with jscodeshift in rule un-jsx SyntaxError: Identifier ‘Z’ has already been declared. (46:13)
    • Failed to parse rule module-34408.js with jscodeshift in rule un-iife SyntaxError: Identifier ‘Z’ has already been declared. (46:13)
    • Failed to parse rule module-34408.js with jscodeshift in rule un-es6-class SyntaxError: Identifier ‘Z’ has already been declared. (46:13)
    • Failed to parse rule module-34408.js with jscodeshift in rule un-async-await SyntaxError: Identifier ‘Z’ has already been declared. (46:13)
  • stage2-unpacked/pages/_app/module-65100.js
    • Error running rule un-parameters-lebab on module-65100.js SyntaxError: Unexpected token $
    • Failed to parse rule module-65100.js with jscodeshift in rule un-jsx SyntaxError: Missing semicolon. (310:15)
    • Failed to parse rule module-65100.js with jscodeshift in rule un-iife SyntaxError: Missing semicolon. (310:15)
    • Failed to parse rule module-65100.js with jscodeshift in rule un-es6-class SyntaxError: Missing semicolon. (310:15)
    • Failed to parse rule module-65100.js with jscodeshift in rule un-async-await SyntaxError: Missing semicolon. (310:15)
    • Error running rule prettier on module-65100.js SyntaxError: Missing semicolon. (310:16)
  • stage2-unpacked/pages/_app/module-80894.js
    • Error running rule un-parameters-lebab on module-80894.js SyntaxError: Identifier ‘Z’ has already been declared
    • Failed to parse rule module-80894.js with jscodeshift in rule un-jsx SyntaxError: Identifier ‘Z’ has already been declared. (103:13)
    • Failed to parse rule module-80894.js with jscodeshift in rule un-iife SyntaxError: Identifier ‘Z’ has already been declared. (103:13)
    • Failed to parse rule module-80894.js with jscodeshift in rule un-es6-class SyntaxError: Identifier ‘Z’ has already been declared. (103:13)
    • Failed to parse rule module-80894.js with jscodeshift in rule un-async-await SyntaxError: Identifier ‘Z’ has already been declared. (103:13)
  • ???
    • Multiple exports of “default” found, only the last one will be kept
  • stage2-unpacked/pages/_app/module-32004.js
    • Error running rule un-parameters-lebab on module-32004.js SyntaxError: Identifier ‘Z’ has already been declared
    • Failed to parse rule module-32004.js with jscodeshift in rule un-jsx SyntaxError: Identifier ‘Z’ has already been declared. (50:13)
    • Failed to parse rule module-32004.js with jscodeshift in rule un-iife SyntaxError: Identifier ‘Z’ has already been declared. (50:13)
    • Failed to parse rule module-32004.js with jscodeshift in rule un-es6-class SyntaxError: Identifier ‘Z’ has already been declared. (50:13)
    • Failed to parse rule module-32004.js with jscodeshift in rule un-async-await SyntaxError: Identifier ‘Z’ has already been declared. (50:13)
  • stage2-unpacked/pages/_app/module-49948.js
    • Error running rule un-parameters-lebab on stage2-unpacked/pages/_app/module-49948.js Error: did not recognize object of type “PropertyDefinition”
  • Error: ENOENT: no such file or directory, open ‘out/perf.json’

Input code

From this commit:

- https://github.com/0xdevalias/chatgpt-source-watch/commit/48088035667d123c839b05871be5cb37de75dd5f
  - `unpacked/_next/static/chunks/pages/_app.js`

Direct file link:

- https://github.com/0xdevalias/chatgpt-source-watch/blob/48088035667d123c839b05871be5cb37de75dd5f/unpacked/_next/static/chunks/pages/_app.js

Reproduction

No response

Steps to reproduce

See above

Expected behavior

Unpack/Unminify works properly.

Actual behavior

The error described above occurs.

About this issue

  • Original URL
  • State: closed
  • Created 6 months ago
  • Comments: 18 (12 by maintainers)

Commits related to this issue

Most upvoted comments

yes, currently looks like this (webpack 5 doesn’t have that many tests because the logic is mostly the same) feel free to expand if you find any more

exports 

describe('webpack 4', () => {
  test('export named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "counter", function() { return foo; });
      var foo = 1;
    `).toMatchInlineSnapshot(`export var counter = 1;`));

  test('export default expression;', () =>
    expectJS(`
      __webpack_exports__.default = 1;
    `).toMatchInlineSnapshot(`export default 1;`));

  // TODO: create separate transform for extracting unrelated variables from for loop?
  test.todo('export inlined variable', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "counter", function() { return foo; });
      for (var foo = 1, i = 0; i < 10; i++) {}
    `).toMatchInlineSnapshot(`
      export var foo = 1;
      for (var i = 0; i < 10; i++) {}
    `));

  test('export default variable', () =>
    expectJS(`
    var foo = 1;
    __webpack_exports__.default = foo;
    `).toMatchInlineSnapshot(`export default 1;`));

  test('export default variable with multiple references', () =>
    expectJS(`
      var foo = 1;
      __webpack_exports__.default = foo;
      console.log(foo);
    `).toMatchInlineSnapshot(`
      var foo = 1;
      export default foo;
      console.log(foo);
    `));

  test('export default function', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "default", function() { return foo; });
      function foo() {}
    `).toMatchInlineSnapshot(`export default function foo() {}`));

  test('export default class', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "default", function() { return foo; });
      class foo {}
    `).toMatchInlineSnapshot(`export default class foo {}`));

  test('re-export named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "readFile", function() { return lib.readFile; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { readFile } from "lib";
    `));

  test('re-export named with multiple references', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "readFile", function() { return lib.readFile; });
      var lib = __webpack_require__("lib");
      console.log(lib);
    `).toMatchInlineSnapshot(`
      import * as lib from "lib";
      export { readFile } from "lib";
      console.log(lib);
    `));

  test('re-export named as named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "foo", function() { return lib.readFile; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { readFile as foo } from "lib";
    `));

  test('re-export named as default', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "default", function() { return lib.readFile; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { readFile as default } from "lib";
    `));

  test('re-export default as named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "foo", function() { return lib.default; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { default as foo } from "lib";
    `));

  test('re-export default as default', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "default", function() { return lib.default; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { default } from "lib";
    `));

  // webpack just declares all the exports individually, hard to detect this case
  test.todo('re-export all'); // export * from 'lib';

  test.todo('re-export all from commonjs', () =>
    expectJS(`
      var lib = __webpack_require__("lib");
      var libDef = __webpack_require__.n(lib);
      for (var importKey in lib) {
        if (["default"].indexOf(importKey) < 0) {
          (function (key) {
            __webpack_require__.d(__webpack_exports__, key, function () {
              return lib[key];
            });
          })(importKey);
        }
      }
    `).toMatchInlineSnapshot(`
      export * from "./lib";
    `),
  );

  test('re-export all as named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "lib", function() { return lib; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export * as lib from "lib";
    `));

  test.todo('re-export all as default', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, "default", function() { return lib; });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`export * as default from "lib";`),
  );

// https://webpack.js.org/plugins/module-concatenation-plugin/, https://github.com/webpack/webpack/blob/dfffd6a241bf1d593b3fd31b4b279f96f4a4aab1/lib/optimize/ConcatenatedModule.js#L58
// almost impossible to separate concatenated modules, so ignore it instead
  test('namespace object', () =>
    expectJS(`
        var lib_namespaceObject = {};
        __webpack_require__.d(lib_namespaceObject, "foo", function() { return foo; });
        function foo() {}
      `).toMatchInlineSnapshot(`
        var lib_namespaceObject = {};
        //webcrack:concatenated-module-export
        Object.defineProperty(lib_namespaceObject, "foo", {
          enumerable: true,
          get: () => foo
        });
        function foo() {}
      `));
});

describe('webpack 5', () => {
  test('export named', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, {
        counter: () => foo
      });
      var foo = 1;
    `).toMatchInlineSnapshot(`
      export var counter = 1;
    `));

  test('export same variable with multiple names', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, {
        counter: () => foo,
        increment: () => foo,
      });
      var foo = 1;
    `).toMatchInlineSnapshot(`
      export var counter = 1;
      export { counter as increment };
    `));

  test.todo('export object destructuring', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, {
        bar: () => bar,
        name1: () => name1
      });
      const o = {
        name1: "foo",
        name2: "bar"
      };
      const {
        name1,
        name2: bar
      } = o;
    `).toMatchInlineSnapshot(`
      const o = {
        name1: "foo",
        name2: "bar"
      };
      export const {
        name1,
        name2: bar
      } = o;
    `),
  );

  test.todo('export array destructuring', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, {
        bar: () => bar,
        name1: () => name1
      });
      const o = ["foo", "bar"];
      const [name1, bar] = o;
    `).toMatchInlineSnapshot(`
      const o = ["foo", "bar"];
      export const [name1, bar] = o;
    `),
  );

  test('re-export named merging', () =>
    expectJS(`
      __webpack_require__.d(__webpack_exports__, {
        readFile: () => lib.readFile,
        writeFile: () => lib.writeFile,
      });
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      export { readFile, writeFile } from "lib";
    `));

  test.todo('re-export all from commonjs', () =>
    expectJS(`
      var lib = __webpack_require__("lib");
      var libDef = __webpack_require__.n(lib);
      var reExportObject = {};
      for (const importKey in lib) {
        if (importKey !== "default") {
          reExportObject[importKey] = () => lib[importKey];
        }
      }
      __webpack_require__.d(__webpack_exports__, reExportObject);
    `).toMatchInlineSnapshot(`
      export * from "./lib";
    `),
  );

  test('namespace object', () =>
    expectJS(`
      var lib_namespaceObject = {};
      __webpack_require__.d(lib_namespaceObject, {
        foo: () => foo,
        bar: () => bar,
      });
      function foo() {}
      function bar() {}
    `).toMatchInlineSnapshot(`
        var lib_namespaceObject = {};
        //webcrack:concatenated-module-export
        Object.defineProperty(lib_namespaceObject, "bar", {
          enumerable: true,
          get: () => bar
        });
        //webcrack:concatenated-module-export
        Object.defineProperty(lib_namespaceObject, "foo", {
          enumerable: true,
          get: () => foo
        });
        function foo() {}
        function bar() {}
      `));
});

imports 

describe('webpack 4', () => {
  test('default import', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log(lib.default);
    `).toMatchInlineSnapshot(`
      import lib from "lib";
      console.log(lib);
    `));

  test('default import of commonjs module', () =>
    expectJS(`
      var lib = __webpack_require__(1);
      var _tmp = __webpack_require__.n(lib);
      console.log(_tmp.a);
      console.log(_tmp());
    `).toMatchInlineSnapshot(`
      import _default from "1";
      console.log(_default);
      console.log(_default);
    `));

  test('inlined default import of commonjs module', () =>
    expectJS(`
      var lib = __webpack_require__(1);
      console.log(__webpack_require__.n(lib).a);
    `).toMatchInlineSnapshot(`
      import _default from "1";
      console.log(_default);
    `));

  test('named import', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log(lib.foo);
    `).toMatchInlineSnapshot(`
      import { foo } from "lib";
      console.log(foo);
    `));

  test('named import with indirect call', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log(Object(lib.foo)("bar"));
    `).toMatchInlineSnapshot(`
      import { foo } from "lib";
      console.log(foo("bar"));
    `));

  test('namespace import', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log(lib);
    `).toMatchInlineSnapshot(`
      import * as lib from "lib";
      console.log(lib);
    `));

  test('side effect import', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      var lib = __webpack_require__("lib");
    `).toMatchInlineSnapshot(`
      import "lib";
    `));

  test.todo('dynamic import', () =>
    expectJS(`
      __webpack_require__.e("chunkId").then(__webpack_require__.bind(null, "lib")).then((lib) => {
        console.log(lib);
      });
    `).toMatchInlineSnapshot(`
      import("lib").then((lib) => {
        console.log(lib);
      });
    `),
  );
});

describe('webpack 5', () => {
  test('named import with indirect call', () =>
    expectJS(`
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log((0, lib.foo)("bar"));
    `).toMatchInlineSnapshot(`
      import { foo } from "lib";
      console.log(foo("bar"));
    `));

  test.todo('namespace import of commonjs module', () =>
    expectJS(`
      var _cache;
      __webpack_require__.r(__webpack_exports__);
      const lib = __webpack_require__("lib");
      console.log(_cache ||= __webpack_require__.t(lib, 2));
    `).toMatchInlineSnapshot(`
      import * as lib from "lib";
      console.log(lib);
    `),
  );

  test.todo('dynamic import', () =>
    expectJS(`
      __webpack_require__.e("chunkId").then(__webpack_require__.bind(__webpack_require__, "lib")).then((lib) => {
        console.log(lib);
      });
    `).toMatchInlineSnapshot(`
      import("lib").then((lib) => {
        console.log(lib);
      });
    `),
  );
});

+3
j4k0xb on Jan 12, 2024

The module.exports = x assignment will overwrite the default export, which means the first line will be dropped. This is either something wrong or wakaru wrong. So a warning was printed to hint to users.

module 19841 has to be converted to one of these:

function eo() { }
export { eo as clsx };
export default eo;
// or
function clsx() { }
export { clsx };
export default clsx;
// or (preferred one)
export function clsx() { }
export default clsx;

there’s no commonjs/mixed export so no module.exports = either, which should get rid of the issue

example for comparison (webpack 4):

export function f() {}
export default f;

__webpack_require__.r(__webpack_exports__);
/* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "f", function() { return f; });
function f() { }
/* harmony default export */ __webpack_exports__["default"] = (f);

I recently bundled all possible import/export/esm/commonjs combinations and we both need to fix quite a bit to do it “properly” 😅

+3
j4k0xb on Jan 9, 2024

https://github.com/lebab/lebab/issues/353

But I might want to re-implement these rules in the long run…

+2
pionxzh on Jan 10, 2024

SyntaxError: Identifier ‘f’ has already been declared.

@0xdevalias This is related to lebab. I’m still working on creating a minimal reproduction. But no more repeated parsing error in CLI 0.0.6.

+2
pionxzh on Jan 9, 2024

All released in CLI 0.0.7

+1
pionxzh on Jan 21, 2024

All errors are fixed. I will move @j4k0xb 's CJS/ESM part to a new issue.

+1
pionxzh on Jan 13, 2024

Error: did not recognize object of type “PropertyDefinition”

It’s caused by the combination of the private field and the numeric comment. I will have to spend some time digging into ast-types 😞 and patch it by myself.

+1
pionxzh on Jan 11, 2024

Thanks for the detailed test cases 🙏 Especially for those re-export cases, I haven’t seen them before.

+1
pionxzh on Jan 9, 2024

lol damm. Thanks. I was so confused about this.

Did you make a test suite or something that we can use as a reference?

+1
pionxzh on Jan 9, 2024

Oh, hm, should early return the result when there is a parsing error.

+1
pionxzh on Jan 9, 2024

Multiple exports of “default” found, only the last one will be kept

I have updated the log to print out the filename.

[module-19841.js] Multiple exports of "default" found, only the last one will be kept

exports.default = eo;

module.exports = {
    clsx: eo
};

The module.exports = x assignment will overwrite the default export, which means the first line will be dropped. This is either something wrong or wakaru wrong. So a warning was printed to hint to users.

BUG: because of the AST scanning order, we actually chose the first one… will fix it

+1
pionxzh on Jan 8, 2024

Error: did not recognize object of type “PropertyDefinition”

This one looks similar to #51, lebab failed to parse the code.

I’m 100% sure about the root cause. It’s quite interesting. If you change the 3e5 to 35, then wakaru won’t generate a /* 3e5 */ comment, then everything works fine 😅

updateGcTime(Y) { 
      this.gcTime = Math.max(
        this.gcTime || 0,
        Y ?? (ei.sk ? 1 / 0 : 3e5) /* 3e5 */
                                     ^^^^ This breaks lebab's parser
      );
    }
+1
pionxzh on Jan 8, 2024

Error running rule un-parameters-lebab on module-33562.js SyntaxError: Identifier ‘Z’ has already been declared

This is caused by the new rule runner. Some scopes are not naturally reset between rules because of the sharing AST. Fixed. This is not easy to cover with normal unit tests. Maybe we can only count on the time and samples.

Error running rule un-parameters-lebab on module-65100.js SyntaxError: Unexpected token $

This is an unhandled case in which backticks and other special characters need to be escaped. Fixed.

+1
pionxzh on Jan 8, 2024

It would be useful if this message mentioned what module was being processed at the time

We will need a logger injected by the context which will automatically print the file and rule name.

+1
pionxzh on Jan 7, 2024
Read more comments on GitHub
← wakaru: CLI bug: Error: ENOENT: no such file or directory, open 'out/perf.json' (when using `--perf` alongside `--unpacker-output` / `--unminify-output`)
Home-Assistant-custom-components-Tauron-AMIplus: 2.4.10: KeyError: 'tauron_importer:<TUTAJ_PUNKT_POBORU>_consumption' →