three.js: Importing examples jsm modules causes bundlers to bundle three.js source code twice

@gkjohnson @donmccurdy

I think I’m all for adopting https://skypack.dev/ and having a script that converts “…/build/three.module.js” to “three” at npm publish time 👍

I think it’s just something to get used to. Now that I think I get it I’m fine with the way it is.

BTW I updated threejsfundamentals to all be esm based so 🤞

Should be fixed with r128! 🎉

This is related to #17220 – one of the solutions proposed there was to replace the main field in package.json with the module build path but that would not fix this use case.

Just to be clear the issue here is that while three is being marked as external to build a separate package that is dependent on three in the rollup config that does not catch the hard reference to ../../../build/three.module.js and includes it in the build. For example building the following file will inadvertently include the OrbitControls code and the threejs code in the bundle, as well as import another copy of three when built with @adrian-delgado’s posted config.

// src/main.js
import * as THREE from 'three';
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls.js';

console.log(THREE, OrbitControls);

@adrian-delgado it might be worth noting that even if the path in OrbitControls.js is changed to three OrbitControls will still be included in your bundle which may or not be desired and could result in at least the OrbitControls code being included twice in dependent applications.

I don’t mean to propose this as a long term or best solution but changing the config to mark OrbitControls (and all files in the three folder) as external would solve this in both cases:

export default {
	// ...

	external: p => /^three/.test(p),

	// ...
};

Please share a minimal reproducible example.

How about having a separate npm package for examples only for end users that has three js as a dependency?

Or to riff on this idea a bit, maybe a pre-publish script that generates a new directory within the existing npm package? It’s only the examples/jsm/* code that needs to be transformed here. In that case we’d have something like:

• examples/js/*: Legacy scripts depending on global THREE.*. For copy/paste workflows.
• examples/jsm/*: ES modules with relative imports from ../../build/three.module.js. For older CDNs and development in three.js repository.
• examples/npm/*: ES modules with bare imports from three. For bundlers, modern CDNs, and Import Maps.

The latter could be generated by a pre-publish script and not checked into the repository.

Could also (needs more research?) set up an exports map so that bundlers automatically resolve a shorthand path, like:

import { OrbitControls } from 'three/controls/OrbitControls.js';

---

It could also have cdn variants that assumes three js has been already imported…

That’s essentially what examples/js/* provides, but it isn’t compatible with ES modules. I don’t think we want to encourage a workflow that mixes global variables with ES modules.

My solution is dumb, but here it is and it works for me:

I use npm to install THREE (thus can do the normal import * as THREE from "three"). That’s not the dump part.

Then, for each of the “examples” that I need (like TrackballControls in your case), I just copy that file into my project and edit it slightly so that it imports ... from "three". That avoids bundling three.js twice. It’s dumb, but it works for my case and it hasn’t caused me any trouble so far. Your mileage may vary.

If the maintainers edited each “example” so that it imports ... from "three" (currently each “example” imports with a relative path instead) then I would not need to copy them into my project to do that edit myself. However, I don’t know the other implications of doing that. Maybe it breaks other peoples’ things I’m not aware of.

@gkjohnson Thanks for the detailed explanation, that makes sense to me.

It sounds like this doesn’t solve the issue in this thread after all, but since I’ve already mentioned it a couple times in the thread, I finally tested out an import map polyfill: https://github.com/KhronosGroup/KTX-Software/pull/172/files. With that polyfill, import * as THREE from 'three'; works in the web browser.

Just to clarify… The examples would continue using “…/build/three.module.js” but because in npm we’d be using “three” people using jsfiddle, codepen, glitch, … would have to use https://skypack.dev/ instead of https://unpkg.com/.

@mrdoob one solution to this might be to make a pre-publish script that converts the import statements in the jsm folder from “…/build/three.module.js” to “three” just for NPM. That way the imports can be bare “three” imports as the npm ecosystem would expect while the files in Github would have proper file path references in them for those that want to download and use the files statically.

I’m not sure how this would affect cdn use like unpkg.com, though. It looks like unpkg.com specifically supports a ?module url parameter to automatically resolve the bare import specifiers but it is marked as experimental on the home page.

Has anyone come up with a solution for this on parcel(v2)?

I’m just doing import * as THREE from 'three/build/three.module' on parcelv2, which seems to be similar to what everyone else is doing. You can get types to work (if they don’t) by using named imports, or by just using import * as THREE from 'three' during development, and switching over to the three.module import later.

+1 that it’d be good if the npm-distributed version of three imported from itself rather than directly referencing a file to import from

@gkjohnson My use case is not exactly the same, since I only want the three lib to be marked as external, not the example (I want the example to be bundled with my build).

@donmccurdy

As far as I know both (1) and (2) are still easy problems to stumble into?

In my opinion (2) is a result of configuring the bundler incorrectly and maybe we can address that by updating the docs with some suggestions for bundlers. (1) can occur as a result from using a package that suffers from problem (2) but other than that I’m not convinced (1) is easy to stumble upon. I’d like to see a real world use case that demonstrates the issue to see how someone configured their bundler but here’s a list of the ways I know that you can hit this (so far):

1. Explicitly import from 'three/src/Three.js', or 'three/build/three.min.js' (which is not recommended in the docs).
2. Reconfigure your bundler to use the package.main field rather than the package.module field when resolving. The three big bundlers Rollup, Webpack, and Parcel all prefer module over main by default, however. This use case feels like it would be uncommon but that’s just an assumption.
3. Use npm link to include a symlinked package that depends on three (this is fixed by using rollup’s preserveSymlinks option)
4. Use three and examples in codesandbox.io because the platform prioritizes the main field over module.

Number 4 seems like the only one that could be stumbled upon easily, though I know people are doing 1 for tree shaking. The others feel like they’re outside of our control or would be very uncommon.

@chabb

as far as I understand, I guess it’s because the examples are doing import from "../../../build/three.module.js"; …

This isn’t the case please read what I’ve explained here: https://github.com/mrdoob/three.js/issues/17482#issuecomment-583694493. /^three works because it matches the string 'three/examples/jsm/controls/OrbitControls.js' which should also be external because it’s part of the three.js library while the string 'three' does not. The same can happen with other dependencies, too. I’d recommend using regex for all dependencies to avoid other unknown pitfalls or match against any package with a bare module specifier.

@chabb

I’m writing a React component using three.js, and i am importing some modules from the examples. Once it’s bundled with rollup, if i look at the bundle, i can see that there is one import statement for three, and then the Three.js code.

The posted solution here should solve your issue: https://github.com/mrdoob/three.js/issues/17482#issuecomment-530957570.

I feel a lot of these issues are derived from people not fully understanding what’s happening with their bundler (which is understandable) but these problems are not unique to three. It’s possible, though, that accidentally double importing three core is just more noticeable than with other libraries. Bundling a dependency that’s intended to be external like lodash, a react component, or OrbitControls can just be more easily missed.

Regarding depending on an external package Rollup documents this behavior and provides an option here and Webpack has a similar option here. In this case if the example files instead referred to “three” then while the core library would not be bundled you’d still get duplicate bundles of example code which is it’s own problem. And I don’t think there’s anything this project can do to help a bundler interpret npm link pitfalls. I think the only problematic case I’ve seen that I feel isn’t the result of a misconfigured bundler is the codesandbox case.

For the bundler cases maybe the answer is to document, add a troubleshooting guide, or link to how to configure the common bundlers on the importing via modules page.

@greggman

Not sure it’s related a similar happens if you try to use modules live like this

import * as three from 'https://cdnjs.cloudflare.com/ajax/libs/three.js/108/three.module.js';
import { OrbitControls } from 'https://threejs.org/examples/jsm/controls/OrbitControls.js';

Yeah… Don’t use modules like that 😁

So today I found myself doing just that… 😅 It’s a bad habit indeed, but the problem is that most things kind of work but if something breaks is pretty hard to nail down.

In my case I was importing three.module.js from dev and OBJLoader from master. OBJLoader imported three.module.js from master so the BufferGeometry didn’t have the new usage property, and WebGLRenderer did not render the mesh because it didn’t find usage, everything else worked though 😶

This is pretty hairy…

To mention the options, other CDNs like jsdelivr or unpkg do support ES modules:

• https://cdn.jsdelivr.net/npm/three@v0.108.0/examples/jsm/loaders/GLTFLoader.js
• https://unpkg.com/three@0.108.0/examples/jsm/loaders/GLTFLoader.js

Not sure it’s related a similar happens if you try to use modules live like this

import * as three from 'https://cdnjs.cloudflare.com/ajax/libs/three.js/108/three.module.js';
import { OrbitControls } from 'https://threejs.org/examples/jsm/controls/OrbitControls.js';

loads three.js twice, once from the CDN and again from threejs.org

Maybe that’s not the way modules are supposed to be used with three but just going from pre 106 there are 1000s of sites and examples that do

<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/105/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>

All the examples show using modules live instead of building(bundling) so in a sense they aren’t showing the actual way to use three.js like they used to. In other words, the old examples worked out of the box. The new examples don’t AFAIK. In order for an example to work you’d need to extract the JavaScript out of the example and put in a separate .js file, then put three.js locally (probably via npm). Fix all the paths in the examples so they are package based paths (no …/…/./build), and finally use rollup

That’s pretty big change from the non module versions for which just changing the paths was enough

Should be fixed with r128! 🎉

Updated the version of "three": "^0.128.0" and "@types/three": "^0.127.1" in package.json. But, still observing the **WARNING: Multiple instances of Three.js being imported**. while running the test.

Question: Does any jest configuration (here) need to be taken care of?

Thanks! Did a bit of clean up 👍

Just to clarify, the current solution doesn’t yet work with https://unpkg.com/ (needs ?module). People should use https://www.skypack.dev/ instead.

Just made PR #21654 with a prepublish script.

@donmccurdy

just to confirm, we’d be transforming the examples/jsm folder at publish time — not creating a 3rd directory

I would be concerned that publishing a third folder would cause confusion with some people pulling from the wrong folder etc.

with the understanding that unpkg and older CDNs won’t work for examples/jsm in future releases?

Which other CDNs will break? Only CDNs that pull directly from NPM will break. Are there others? I would think that most CDNs that pull from NPM should be bare-module aware considering that’s the predominant way to distribute packages there. I like unpkg but I expect it to break in a lot of cases like the example I posted above and I think it’s a problem that it doesn’t support bare modules more robustly.

Could we do an “alpha” release with these changes first, to confirm everything works as expected on skypack?

What are you thinking might break? I’m not against a test run of some kind but I’m also thinking that if it doesn’t work for some reason a point release could be made with reverted paths.

Could also (needs more research?) set up an exports map so that bundlers automatically resolve a shorthand path, like:

I like this idea. It looks like we could just add this to package.json:

"exports": {
    "./": "./examples/jsm/"
}

How about having a seperate npm package for examples only for end users that has three js as a dependency?

It could also have cdn variants that assumes three js has been already imported…

This possibly solves the problem: rather than the default Three.js, specifically import the same module as OrbitControls uses:

import * as THREE from 'https://cdn.skypack.dev/three@0.120.0/build/three.module.js'
import { OrbitControls } from 'https://cdn.skypack.dev/three@0.120.0/examples/jsm/controls/OrbitControls.js'

I presume that is similar to what the rest of us are doing? I.e. the equivalent of:

import * as THREE from '../node_modules/three/build/three.module.js'
import { OrbitControls } from '../node_modules/three/examples/jsm/controls/OrbitControls.js'

Two cases here:

(1) user wants threejs and examples included once, gets something twice

E.g. while building an application.

(2) user wants threejs and examples included zero times, gets something 1+ times

E.g. while building a library with three as an external or peer dependency.

---

As far as I know both (1) and (2) are still easy problems to stumble into? If the approach above solves (1), that alone is helpful. I’m not sure about (2). Maybe the /^three/.test( path ) trick should be mentioned on import via modules?

@donmccurdy

I have a hunch that if examples/jsm packages could change this pattern… these issues would be much easier to resolve.

I think this would make it looks resolved but people would still have duplicated code that’s just harder to notice becaues it doesn’t cause the application to break.

I don’t quite follow this. Because they’re relative path imports? We could make them package-relative.

Sorry if I’m unclear I think this is a bit difficult to explain – hopefully this is a bit more clear. I’ll use the Rollup case:

In the cases above where people want to rollup a package with three marked as external I assume they’re building a library where three.js would be a peer dependency that another application could rely on:

import * as THREE from 'three';
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls.js';
import { stuff } from './local/src/index.js';

// library code with exports...

Here the goal would be for the above three.js imports to remain in the library and the bundle to load three and OrbitControls as peer dependencies so if the application also uses three.js and OrbitControls you don’t import either twice.

People expect the option external: [ 'three' ] to achieve this behavior for them (I certainly did) but it doesn’t because the string doesn’t match the OrbitControls import path. This results in OrbitControls being unintentionally bundled and therefore ../../../build/three.module.js being bundled, as well (because it also doesn’t match the string). I think people point to the three.js core file being bundled because it’s much more noticeable – the applications break, the library bundle is much larger, etc – where the reality is that the OrbitControls file shouldn’t have been bundled in the first place. The correct way to configure Rollup here is to set the option to external: path => /^three/.test( path ).

This isn’t unique to three. Rollup uses lodash as an example in its docs but it’s going to be hard / impossible to notice if 'lodash/merge' gets bundled in your library code because it’s so small and won’t cause duplicate import bugs. Material UI encourages nested file references in imports and likewise the setting external: ['@material-ui/core'] would fail to exclude '@material-ui/core/Button' from the bundle.

I don’t think it’s worthwhile to change the example code for these use cases because it will still result in duplicate code that wouldn’t be there if the bundler was configured properly.

I have a hunch that if examples/jsm packages could change this pattern…

// @file GLTFLoader.js

// Before
import { Mesh } from '../../build/three.module.js';

// After
import { Mesh } from 'three';

… these issues would be much easier to resolve. Unfortunately, I don’t know how we’d manage the HTML examples within the threejs website without a complex build setup then. An import map polyfill on the threejs website might solve it, but I’m not sure. 😕

if the example files instead referred to “three” then while the core library would not be bundled you’d still get duplicate bundles of example code…

I don’t quite follow this. Because they’re relative path imports? We could make them package-relative.

I’ve developed a plugin three-minifier which may help solve this problem.

It does kind of seem like it might be good to have a three.module.min.js though (or is that three.min.module.js 😜)

One idea that would take some investigation, but could be interesting… if we’d be willing to add an import map polyfill to all the examples, I think we could make the imports used there 100% copy/paste compatible with npm- and bundler-based workflows. For example:

<script defer src="es-module-shims.js"></script>
<script type="importmap-shim" src="importmap.dev.js"></script>

<!-- ... -->

<script type="module-shim">
  import { Scene, WebGLRenderer } from 'three';
  import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';
  
  // ...
</script>

Yeah… Don’t use modules like that 😁

Agreed. It’s just arguably the docs and examples are targeting mostly inexperienced developers and the fact that jsm examples are the default and none of them will work without a builder nor will they work via any CDN is a kind of big change.

It used to be you could basically view source on an example, copy and paste into jsfiddle/codepen etc, fix the paths in the script tags and it would run. Now though all the examples won’t run unless you link directly into the three.js site and watch them break each time the version gets bumped. (yes I know the non module examples exist but those are not the ones linked to from https://threejs.org/examples)

@gkjohnson

import * as three from 'https://cdnjs.cloudflare.com/ajax/libs/three.js/108/three.module.js';
import { OrbitControls } from 'https://cdnjs.cloudflare.com/ajax/libs/three.js/108/examples/jsm/controls/OrbitControls.js';

Doesn’t work, OrbitControls are not on the CDN and the paths inside the OrbitContrls …/…/…/bulild/three.js is not the correct path to make it work

// or

import * as three from 'https://threejs.org/build/three.module.js';
import { OrbitControls } from 'https://threejs.org/examples/jsm/controls/OrbitControls.js'

Also doesn’t work as it will break every time three.js pushes a new version

Maybe push the examples/js folder to a CDN and three such that just fixing the urls in the example code will still work? That means three.module.js needs to be at

https://cdnjs.cloudflare.com/ajax/libs/three.js/108/build/three.module.js

build added to the path

If you treat three as external dependency:

export default {
	input: 'src/main.js',
	external: ['three'],
	output: [
		{
			format: 'umd',
			name: 'LIB',
			file: 'build/main.js'
		}
	],
	plugins: [ resolve() ]
};

then the output should not contain the source code of three.js, yet it includes everything.

If, however, you don’t import the OrbitControls, then the output will only include the source code of the main.js file.

you can try it out by commenting out the OrbitControls import, and then building again (but with 'three' as external dependency).