webpack: Webpack is producing invalid sourcemaps?

tl;dr

This issue was open before and marked as fixed https://github.com/webpack/webpack/issues/7616, but unfortunately, it’s still reproducible with the latest version of webpack
According to https://sourcemaps.io/ and sourcemap-validator, the sourcemaps produced by webpack are invalid
using devtool: "source-map", a barebones webpack config, and webpack@4.23.1
sample repro repo available at https://github.com/adaniliuk/webpack-sourcemap-testing (this might be an issue with sourcemap-validator, but the error that it’s throwing (in the “current behavior” section below) seems pretty legit)

Bug report

What is the current behavior?

Running webpack with the following config:

const webpack = require('webpack');
const path = require('path');

module.exports = {
    entry: './src/index.js',
    output: {
        path: path.resolve(__dirname, 'dist'),
        filename: 'bundle.js'
    },
    devtool: 'source-map'
};

produces dist/bundle.js and dist/bundle.js.map.
running those files through sourcemap-validator (or https://sourcemaps.io/, and consequently Sentry) produces the following error:

/webpack-sourcemap-testing/node_modules/sourcemap-validator/index.js:146
          throw new Error(errMsg);
          ^
Error: Warning: mismatched names
Expected: installedModules || 'installedModules' || 'installedModules' || "installedModules" || "installedModules"
Got:  	var installedM ||  	var installedMod ||  	var installedMod ||  	var installedMod ||  	var installedMod
Original Line:  	var installedModules = {};
Mapping: 1:17→2:0 "installedModules" in webpack:///webpack/bootstrap
    at /webpack-sourcemap-testing/node_modules/sourcemap-validator/index.js:146:17
    at Array.forEach (<anonymous>)
    at SourceMapConsumer_eachMapping [as eachMapping] (/webpack-sourcemap-testing/node_modules/sourcemap-validator/node_modules/source-map/lib/source-map/source-map-consumer.js:570:10)
    at validate (/webpack-sourcemap-testing/node_modules/sourcemap-validator/index.js:83:12)
    at Object.<anonymous> (/webpack-sourcemap-testing/test-sourcemaps.js:6:1)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)

If the current behavior is a bug, please provide the steps to reproduce.

Repro repo here: https://github.com/adaniliuk/webpack-sourcemap-testing

Repro instructions for that repo are on the readme.

If starting from scratch, use the following webpack config:

const webpack = require('webpack');
const path = require('path');

module.exports = {
    entry: './src/index.js',
    output: {
        path: path.resolve(__dirname, 'dist'),
        filename: 'bundle.js'
    },
    devtool: 'source-map'
};

and use sourcemap-validator to test dist/bundle.js for validity.

What is the expected behavior?

webpack produces valid source maps
so sourcemap-validator reads the sourcemap as valid (e.g., not throw an error).

It looks like it might have something to do with some misaligned whitespace, but I’m really not sure.

Other relevant information:

webpack version: 4.23.1 Node.js version: 8.11.3 Operating System: macOS 10.14 (18A391) Additional tools: sourcemap-validator@1.1.0, yarn@1.9.2

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 12
Comments: 45 (5 by maintainers)

Commits related to this issue

Use eval-source-map in prod mode Fixes an issue where invalid prod mode source maps cause browsers to be unable to open the client when built in prod mode. See https://github.com/webpack/webpack/iss... — committed to andrewazores/cryostat-web by andrewazores 4 years ago
Use eval-source-map in prod mode (#68) Fixes an issue where invalid prod mode source maps cause browsers to be unable to open the client when built in prod mode. See https://github.com/webpack/we... — committed to cryostatio/cryostat-web by andrewazores 4 years ago
Use eval-source-map in prod mode (#68) Fixes an issue where invalid prod mode source maps cause browsers to be unable to open the client when built in prod mode. See https://github.com/webpack/we... — committed to andrewazores/cryostat-web by andrewazores 4 years ago
Merge React rewrite into master (#81) * Initial rewrite commit using patternfly-react-seed * Initial port of command channel service * Initial port of API service * Very rough auth flow supp... — committed to cryostatio/cryostat-web by andrewazores 4 years ago

Most upvoted comments

Any updates on this?

Still get invalid sourcemaps with v4.29.6

theshem on Apr 2, 2019

Any update on this recently?

september28 on Jun 28, 2019

Invalid sourcemaps with v4.28.1

jlchereau on Jan 9, 2019

@kenhoff thanks, need investigate who break source map 😕

alexander-akait on Dec 27, 2018

I just tried the same scripts installed on a Windows 10 machine to rule out the system architecture. Still no joy. Is anyone from Webpack tracking this issue? It is pretty fundamental to JS projects in Webpack…

september28 on Jun 3, 2019

Okay, I’ve got some news. Let me describe my debugging approach first.

My config:

const path = require('path');

module.exports = {
    entry: './index.js',
    output: {
        path: path.resolve(__dirname, 'dist'),
        filename: 'bundle.js'
    },
    devtool: 'source-map'
};

index.js is just console.log(123);.

I use ndb to step through the code. I also used surge.sh to quickly test against the source map validator (I know there’s a npm package too, but this was faster). Some modifications I made (I wish there were a tool to diff a dirty node_modules!):

./node_modules/terser/package.json

Change main to dist/bundle.js, and delete ./node_modules/terser/dist/bundle.js.map. The first part will load the non-minified module (so its readable in the debugger) and the second part will prevent ndb from transforming stack frames to the original terser source files (because I want to modify dist/bundle.js).

./node_modules/terser-webpack-plugin/dist/minify.js

Add console.log(inputSourceMap.sourcesContent[0]); ~L161. This gives a nice look at the input source map provided by Webpack to Terser. Output:

Click to expand

 	// The module cache
 	var installedModules = {};

 	// The require function
 	function __webpack_require__(moduleId) {

 		// Check if module is in cache
 		if(installedModules[moduleId]) {
 			return installedModules[moduleId].exports;
 		}
 		// Create a new module (and put it into the cache)
 		var module = installedModules[moduleId] = {
 			i: moduleId,
 			l: false,
 			exports: {}
 		};

 		// Execute the module function
 		modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);

 		// Flag the module as loaded
 		module.l = true;

 		// Return the exports of the module
 		return module.exports;
 	}


 	// expose the modules object (__webpack_modules__)
 	__webpack_require__.m = modules;

 	// expose the module cache
 	__webpack_require__.c = installedModules;

 	// define getter function for harmony exports
 	__webpack_require__.d = function(exports, name, getter) {
 		if(!__webpack_require__.o(exports, name)) {
 			Object.defineProperty(exports, name, { enumerable: true, get: getter });
 		}
 	};

 	// define __esModule on exports
 	__webpack_require__.r = function(exports) {
 		if(typeof Symbol !== 'undefined' && Symbol.toStringTag) {
 			Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' });
 		}
 		Object.defineProperty(exports, '__esModule', { value: true });
 	};

 	// create a fake namespace object
 	// mode & 1: value is a module id, require it
 	// mode & 2: merge all properties of value into the ns
 	// mode & 4: return value when already ns object
 	// mode & 8|1: behave like require
 	__webpack_require__.t = function(value, mode) {
 		if(mode & 1) value = __webpack_require__(value);
 		if(mode & 8) return value;
 		if((mode & 4) && typeof value === 'object' && value && value.__esModule) return value;
 		var ns = Object.create(null);
 		__webpack_require__.r(ns);
 		Object.defineProperty(ns, 'default', { enumerable: true, value: value });
 		if(mode & 2 && typeof value != 'string') for(var key in value) __webpack_require__.d(ns, key, function(key) { return value[key]; }.bind(null, key));
 		return ns;
 	};

 	// getDefaultExport function for compatibility with non-harmony modules
 	__webpack_require__.n = function(module) {
 		var getter = module && module.__esModule ?
 			function getDefault() { return module['default']; } :
 			function getModuleExports() { return module; };
 		__webpack_require__.d(getter, 'a', getter);
 		return getter;
 	};

 	// Object.prototype.hasOwnProperty.call
 	__webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); };

 	// __webpack_public_path__
 	__webpack_require__.p = "";


 	// Load entry module and return exports
 	return __webpack_require__(__webpack_require__.s = 0);

./node_modules/terser/dist/bundle.js

I set a conditional breakpoint in do_add_mapping:

if (mapping.token.value === 'installedModules') debugger;

The token col is 14, which is unexpected because input source map printed above shows less indentation for this line - it should be something like 7. (I’m unclear on 0/1 index or if it includes the column before or of the first character of the token… so… I expect 7-ish).

I want to see how this token / AST is made, so I step up a bit and see where Terser parses it. I added a debugger statement around L20578, to inspect the AST and confirm nothing is modifying it between here and do_add_mapping. I inspect toplevel and find the node for installedModules.

toplevel.body[0].body.expression.body[0].definitions[0].name === the installedModules node
toplevel.body[0].body.expression.body[0].definitions[0].name.start.col === 14

Also 14, which suggests I should stop looking inside Terser for the issue. I notice that the file input has a bunch of /******/ prefixes on each line.

Click to expand

"/******/ (function(modules) { // webpackBootstrap
/******/// The module cache
/******/var installedModules = {};
/******/
/******/// The require function
/******/function __webpack_require__(moduleId) {
/******/
/******/	// Check if module is in cache
/******/	if(installedModules[moduleId]) {
/******/		return installedModules[moduleId].exports;
/******/	}
/******/	// Create a new module (and put it into the cache)
/******/	var module = installedModules[moduleId] = {
/******/		i: moduleId,
/******/		l: false,
/******/		exports: {}
/******/	};
/******/
/******/	// Execute the module function
/******/	modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
/******/
/******/	// Flag the module as loaded
/******/	module.l = true;
/******/
/******/	// Return the exports of the module
/******/	return module.exports;
/******/}
/******/
/******/
/******/// expose the modules object (__webpack_modules__)
/******/__webpack_require__.m = modules;
/******/
/******/// expose the module cache
/******/__webpack_require__.c = installedModules;
/******/
/******/// define getter function for harmony exports
/******/__webpack_require__.d = function(exports, name, getter) {
/******/	if(!__webpack_require__.o(exports, name)) {
/******/		Object.defineProperty(exports, name, { enumerable: true, get: getter });
/******/	}
/******/};
/******/
/******/// define __esModule on exports
/******/__webpack_require__.r = function(exports) {
/******/	if(typeof Symbol !== 'undefined' && Symbol.toStringTag) {
/******/		Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' });
/******/	}
/******/	Object.defineProperty(exports, '__esModule', { value: true });
/******/};
/******/
/******/// create a fake namespace object
/******/// mode & 1: value is a module id, require it
/******/// mode & 2: merge all properties of value into the ns
/******/// mode & 4: return value when already ns object
/******/// mode & 8|1: behave like require
/******/__webpack_require__.t = function(value, mode) {
/******/	if(mode & 1) value = __webpack_require__(value);
/******/	if(mode & 8) return value;
/******/	if((mode & 4) && typeof value === 'object' && value && value.__esModule) return value;
/******/	var ns = Object.create(null);
/******/	__webpack_require__.r(ns);
/******/	Object.defineProperty(ns, 'default', { enumerable: true, value: value });
/******/	if(mode & 2 && typeof value != 'string') for(var key in value) __webpack_require__.d(ns, key, function(key) { return value[key]; }.bind(null, key));
/******/	return ns;
/******/};
/******/
/******/// getDefaultExport function for compatibility with non-harmony modules
/******/__webpack_require__.n = function(module) {
/******/	var getter = module && module.__esModule ?
/******/		function getDefault() { return module['default']; } :
/******/		function getModuleExports() { return module; };
/******/	__webpack_require__.d(getter, 'a', getter);
/******/	return getter;
/******/};
/******/
/******/// Object.prototype.hasOwnProperty.call
/******/__webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); };
/******/
/******/// __webpack_public_path__
/******/__webpack_require__.p = "";
/******/
/******/
/******/// Load entry module and return exports
/******/return __webpack_require__(__webpack_require__.s = 0);
/******/ })
/************************************************************************/
/******/ ([
/* 0 */
/***/ (function(module, exports) {

console.log(123);


/***/ })
/******/ ]);

A search for /******/ brings us back to MainTemplate.js.

Consider this code:

Perhaps the issue lies with how ConcatSource / PrefixSource combines maps?

Next, I remove all the extraneous prefixes. If the original code is just:

var installedModules = {};

then I’d expect the original column to be 4 (or 5?).

I remove the PrefixSource and just add bootstrapSource directly. The original column is now 6. The error from source-map-validator is unchanged (In webpack:///webpack/bootstrap: Expected installedModules but got var installedM at L2:0).

I remove the \t on ~L443 of MainTemplate.js and now the original column is 4. This does change the error from source-map-validator, but it doesn’t seem meaningful (In webpack:///webpack/bootstrap: Expected installedModules but got var installedMod at L2:0);

I start trashing some stuff. I minimize the MainTemplate render to just this, while maintaining the error:

const source = new ConcatSource();
// so webpack doesn't remove the dead code
source.add("window.blah = ");
source.add(
	this.hooks.modules.call(
		new RawSource(""),
		chunk,
		hash,
		moduleTemplate,
		dependencyTemplates
	)
);
return source;

In webpack:///index.js: Expected log but got con at L1:0

I print out the map here (console.log(source.map());) and see that the mappings are ;;;;AAAA - but the end result mappings are 2BAAAA,QAAAC,IAAA. There are no validator errors when using ;;;;AAAA. Maybe Webpack makes some more modifications before sending off to minify? I go back to ./node_modules/terser-webpack-plugin/dist/minify.js to check out the input source map given to Terser - it’s still ;;;;AAAA. The map Terser returns is 2BAAAA,QAAAC,IAAA.

Remember when I said I stopped looking for the issue in Terser? Well, I guess that’s busted now.

I’ll take a break at this point 😃

connorjclark on Jun 10, 2019

Why was this closed? Is this fixed? WHere?

stoefln on Jan 15, 2021

Just for the record. We have found out that in our case issue was in the optimizations CDN did to JS bundles before sending them to the clients. So source code didn’t match the map.

Though the above issue still exists and source map validation fails.

adaniliuk on Dec 28, 2018

I believe column mapping is configurable in source map generation (maps are smaller if you don’t embed column info). Maybe there’s a misconfiguration in some tool?

@september28 I’ve set aside time this weekend to dive deeper into this topic. It’s bugged me for a long time too.

connorjclark on Jun 6, 2019

Invalid source-maps with 4.29.6 and 4.30.0.

devtool: ‘source-map’ babel-loader babel/preset-env

Although it doesn’t seem to have anything to do with babel, since I get invalid source maps (wrong line number) even when removing babel completely.

Edit: For now I’m using eval-source-map, since this seems to work.

christophpre on Apr 24, 2019