gatsby: Import d3 causes webpack dependency errors

Adding the line import * as d3 from 'd3' to any JS file in my src code causes the following errors:

ERROR  Failed to compile with 2 errors

These dependencies were not found:

* child_process in ./~/xmlhttprequest/lib/XMLHttpRequest.js
* fs in ./~/xmlhttprequest/lib/XMLHttpRequest.js

To install them, you can run: npm install --save child_process fs

Could this be a bug caused by Gatsby’s webpack config? I don’t understand what’s going on here: Why is XMLHttpRequest throwing an error when it tries to require built in node modules like fs and child_process, and why would these errors be triggered by including d3?

More info

gatsby-config.js

module.exports = {
  siteMetadata: {
    title: 'NathanShane.me',
    author: 'Nathan Shane',
    description: require('./package.json').description
  },
  plugins: [
    'gatsby-plugin-react-helmet',
    'gatsby-plugin-catch-links',
    'gatsby-transformer-sharp',
    'gatsby-plugin-offline',
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/src/pages/projects`,
        name: 'projects',
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/src/pages/blog`,
        name: 'blogPosts',
      },
    },
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: [
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 590
            }
          },
          'gatsby-remark-copy-linked-files'
        ]
      }
    }
  ]
}

gatsby-node.js:

const {resolve} = require('path')

exports.modifyBabelrc = ({ babelrc }) => {
  return {
    plugins: babelrc.plugins.concat([
      ['babel-plugin-root-import']
    ]),
  }
}

exports.createPages = ({boundActionCreators, graphql}) => {
  const {createPage} = boundActionCreators

  return graphql(
    `
    {
      allMarkdownRemark(limit: 1000) {
        edges {
          node {
            fileAbsolutePath
            frontmatter {
              slug
            }
          }
        }
      }
    }
    `
  ).then((result) => {
    if (result.errors) {
      return Promise.reject(result.errors)
    }
    const nodes = result.data.allMarkdownRemark.edges

    const filterByPathIncludes = (testStr, nodes) => (
      nodes.filter(
        ({node: {fileAbsolutePath}}) => (fileAbsolutePath.includes(testStr))
      )
    )

    const projects = filterByPathIncludes('/pages/projects/', nodes)

    projects.forEach(({node}) => {
      const {slug} = node.frontmatter
      createPage({
        path: `/projects/${slug}`,
        component: resolve('./src/templates/project.js'),
        context: {
          slug
        }
      })
    })

    const blogPosts = filterByPathIncludes('/pages/blog/', nodes)

    blogPosts.forEach(({node}) => {
      const {slug} = node.frontmatter
      createPage({
        path: `/blog/${slug}`,
        component: resolve('./src/templates/blogPost.js'),
        context: {
          slug
        }
      })
    })
  }).catch((error) => {
    console.log(error)
  })
}

package.json:

{
  "name": "nathan_shane_site",
  "description": "Nathan Shane's portfolio website",
  "repository": "https://github.com/nwshane/nwshane.github.io.git",
  "license": "MIT",
  "scripts": {
    "dev": "gatsby develop",
    "build": "gatsby build",
    "start": "gatsby serve"
  },
  "dependencies": {
    "babel-plugin-root-import": "^5.1.0",
    "d3": "^4.10.2",
    "gatsby": "^1.8.11",
    "gatsby-link": "^1.4.1",
    "gatsby-plugin-catch-links": "^1.0.8",
    "gatsby-plugin-offline": "^1.0.9",
    "gatsby-plugin-react-helmet": "^1.0.6",
    "gatsby-remark-copy-linked-files": "^1.5.7",
    "gatsby-remark-images": "^1.5.10",
    "gatsby-source-filesystem": "^1.4.12",
    "gatsby-transformer-remark": "^1.7.6",
    "gatsby-transformer-sharp": "^1.6.5",
    "normalize.css": "^7.0.0",
    "ramda": "^0.24.1",
    "styled-components": "^2.1.2",
    "webfontloader": "^1.6.28"
  }
}

Repo and branch in which error is occuring: https://github.com/nwshane/nwshane.github.io/tree/blog

Gatsby version: 1.9.21 NodeJS version: 8.4.0 OS version: Mac OS X 10.12

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 29 (17 by maintainers)

Most upvoted comments

d3-request deleted a field called browser that used to direct webpack to the correct non-node index file. Now d3-request expects the module field to be used.

Commit: https://github.com/d3/d3-request/commit/d635b894fee995930f7419189255da8e66062710#diff-b9cfc7f2cdf78a7f4b91a753d10865a2L19

Solution:

Add this to the webpack config:

{
   resolve: {
      packageMains: [
        'module', // adds check for 'module'
        'webpack',
        'browser',
        'web',
        'browserify',
        ['jam', 'main'],
        'main',
    ]
   }
}

See packageMains config documentation

I’m getting a similar error with fs being required by dotenv, which I presume showed up after reinstalling my node modules a couple of days ago, but I just noticed it today:

 ERROR  Failed to compile with 1 errors 

This dependency was not found:

* fs in ./~/dotenv/lib/main.js

To install it, you can run: npm install --save fs

@mattjstar’s fix works for me too.

EDIT: No it didn’t - it stopped the error messages from appearing, but then dotenv stopped working.

Some findings:

  • d3 requires d3-request code
  • d3-request requires xmlhttprequest code
  • xmlhttprequest requires child-process and fs code
  • d3-request builds two version - one for node and one min.
  • The default export is the node version.
  • Changing the main module of d3-request from build/d3-request.node.js --> build/d3-request.min.js fixes the import.
  • Using the modifyWebpackConfig to include: node: { fs: 'empty', child_process: 'empty' } fixes the issue.

Strange though because this issue seems to not appear when using a very plain webpack.config.js. Trying to find out what webpack differences are triggering the error.

yep, getting the same issue as @hermionewy when using @haroldtreen 's solution with packageMains. iTerm output:

$ gatsby develop
success delete html files from previous builds — 0.010 s
success open and validate gatsby-config.js — 0.003 s
info One or more of your plugins have changed since the last time you ran Gatsby. As
a precaution, we're deleting your site's cache to ensure there's not any stale
data
success copy gatsby files — 0.013 s
success onPreBootstrap — 0.004 s
success source and transform nodes — 0.015 s
success building schema — 0.064 s
success createLayouts — 0.024 s
success createPages — 0.004 s
success createPagesStatefully — 0.011 s
success onPreExtractQueries — 0.001 s
success update schema — 0.044 s
success extract queries from components — 0.057 s
success run graphql queries — 0.013 s
success write out page data — 0.003 s
success write out redirect data — 0.001 s
success onPostBootstrap — 0.000 s

info bootstrap finished - 1.505 s

error There was an error compiling the html.js component for the development server.

See our docs page on debugging HTML builds for help https://goo.gl/yL9lND


  WebpackError: Unexpected token import






  - reactProdInvariant.js:34 Compilation.<anonymous>
    ~/react/lib/reactProdInvariant.js:34:1

  - develop-static-entry.js:3 Compilation.next
    .cache/develop-static-entry.js:3:1



  - reactProdInvariant.js:30 Compilation.<anonymous>
    ~/react/lib/reactProdInvariant.js:30:1

  - develop-static-entry.js:3 Compilation.next
    .cache/develop-static-entry.js:3:1

  - React.js:127 ExtractTextPlugin.<anonymous>
    ~/react/lib/React.js:127:1


  - React.js:78 Object.async.forEachOf.async.eachOf
    ~/react/lib/React.js:78:1

  - React.js:51 Object.async.forEach.async.each
    ~/react/lib/React.js:51:1

  - React.js:79 ExtractTextPlugin.<anonymous>
    ~/react/lib/React.js:79:1


error Command failed with exit code 1.

I was able to get it to work with this, but I’m afraid that will have unintended consequences down the line.

exports.modifyWebpackConfig = ({ config, stage }) => {
  config.merge({
     node: { fs: 'empty', child_process: 'empty' },
  })

  return config;
};