gatsby: createRemoteFileNode can't deal with special characters

Description

In some cases localFile was null, when pulling media from wordpress.

Steps to reproduce

I ran into the issue, while working with gatsby-source-wordpress:

export const fragment = graphql`
  fragment imageTextCombination on WordPressAcf_imageTextCombination {
    image {
      localFile {
        childImageSharp {
          fluid(maxWidth: 500) {
            src
            srcSet
          }
        }
      }
    }
  }
`;

In some cases image.localFile was null which made me wonder. After some time I found out, that the difference in the files appeared to be an ä (a german Umlaut), which is currently not normalized by default in wordpress.

Installing a plugin to normalize fileNames on upload fixed the issue.

I think the root of the issues comes from here:

https://github.com/gatsbyjs/gatsby/blob/1257a225de0bb3be8314ecc5684d1c6635d3ee05/packages/gatsby-source-filesystem/src/create-remote-file-node.js#L344-L348

I’ve checked valid-url and added a test:

t.ok(is_uri('http://localhost/ä'), 'http://localhost/ä');

which would fail.

Expected result

image.localFile should contain the file and not care about the fileName at all. I’m not sure whether I’m right on this, that’s why I opened an issue instead of an PR. What do you think?

Actual result

image.localFile is null.

Environment

  System:
    OS: macOS 10.14
    CPU: x64 Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
    Shell: 5.7.1 - /usr/local/bin/zsh
  Binaries:
    Node: 11.11.0 - /usr/local/bin/node
    Yarn: yarn install v0.21.3
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 47.06s. - /usr/local/bin/yarn
    npm: 6.9.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 69.0.3497.100
    Firefox: 62.0.3
    Safari: 12.0
  npmPackages:
    gatsby: ^2.2.1 => 2.2.1
    gatsby-image: ^2.0.34 => 2.0.34
    gatsby-link: ^2.0.16 => 2.0.16
    gatsby-plugin-react-helmet: ^3.0.10 => 3.0.10
    gatsby-plugin-react-svg: ^2.1.1 => 2.1.1
    gatsby-plugin-sharp: ^2.0.29 => 2.0.29
    gatsby-plugin-styled-jsx: ^3.0.5 => 3.0.5
    gatsby-source-filesystem: ^2.0.27 => 2.0.27
    gatsby-source-wordpress: ^3.0.47 => 3.0.47
    gatsby-transformer-sharp: ^2.1.17 => 2.1.17
  npmGlobalPackages:
    gatsby-cli: 1.1.58

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 15 (10 by maintainers)

Most upvoted comments

I had the same issues with filenames from a S3 bucket with accented characters:

example of a problematic filename téléphonefixe.jpg error message :

here was an error in your GraphQL query:

Unexpected error value: "url passed to createRemoteFileNode is either missing or not a proper web uri: (...)-téléphonefixe.jpg"

I tried to use encodeURI

          return createRemoteFileNode({
            url: encodeURI(source.image.url),
            store,
            cache,
            createNode,
            createNodeId,
            reporter
          });

but then got another error because resulting filename seems to be badly encoded (double encoded ?):

t%25C3%25A9l%25C3%25A9phonefixe.jpg

It supposed to be t%C3%A9l%C3%A9phonefixe.jpg …

PS: @daniloef i’m using a combination of https://www.npmjs.com/package/sanitize-filename and https://www.npmjs.com/package/slug to create clean filenames on upload

yann-yinn on Aug 29, 2020

Thank you, that was quick!

domvo on May 7, 2019

@sidharthachatterjee I think you are right. After reading a bit about the difference of IRI and URI today, I also think the regex shouldn’t be part of the authors is_iri function.

However - the author hasn’t replied to any issues since 2015 or commited anything since then. Similar issues are open already, so maybe the only thing that could convince him is, that Gatsby is so popular. I’ll try 😃

gu-stav on Mar 20, 2019