webpack: perf problems with `require.context`

@sokra this issue is very much alive in webpack5 and also affects dynamic require() and import() calls. I created another repro, although probably @ericpitcher’s is just as good: https://github.com/tmeasday-shopify/webpack-require-context-repro

We are very interested about this for Storybook in large code bases because we rely on require.context() or some other globbing mechanism to import story files.

Would you like me to open a new ticket or can this ticket be reopened?

_Originally posted by @tmeasday in https://github.com/webpack/webpack/issues/6093#issuecomment-868293507_

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 34
  • Comments: 17 (8 by maintainers)

Most upvoted comments

@alexander-akait, sure instead of doing

module.exports = {
    stories: [
        './src/**/*.stories.ts',
    ]
};

I am doing something like this. I oversimplified my code just for this example.

const { sync } = require('glob');

module.exports = {
    stories: sync('./src/**/*.stories.ts')
};

please assign issu to me. i want to work on it .

In theory you can create entry.js file and apply loader, inside loader run glob and create imports like:

import oneStory from "./one-story/index.js";
import twoStory from "./two-story/index.js";

inside loader, use glob-parent (i.e. extract src directory) and use this.addContextDependency https://webpack.js.org/api/loaders/#thisaddcontextdependency, it can be some slow on initial run, but will be fast for incremental builds, also it will tract all changes inside src directory, i.e. we just generate entry file based on glob.

But why do not use https://webpack.js.org/configuration/experiments/#experimentslazycompilation with import()? All stories can be loaded using import(). If you need to add new story just add new lines to your source file with stories. This is actually very close to how it works in browser, even more it can work in browser without bundlers (in theory). WE don’t have glob/require.context in browser and we should migrate from these hacky workarounds.

I believe there’s a developer experience caveat – In my tests, the technique described above has an undesirable side effect: the glob function generates an array of files at start time, and after that, it won’t dynamically index newly added/renamed files.

This means developers need to restart Storybook whenever they add a new <filename>.stories.ts file, as the newly glob-generated array will then contain this new file.

Ideally, we’d have something fast, with the same “dynamic globbing” as before.

@Nantris Can you create reproducible test repo, I will look why it happens

@kaelig, yes you are right. I have this caveat too. I didn’t realized that by doing this, storybook on webpack rebuild does not scan new stories files. I don’t have a better solution for now. Guess this issue will have to be resolved if you want new stories files to be indexed. I am sorry.

@vidal7 Can you provide example?