gatsby: Tests failing since a few weeks ago

Description

While following the guide at https://www.gatsbyjs.org/contributing/setting-up-your-local-dev-environment/ I hit failing tests at step 6. Master at time of writing is ea117e3d8

Two issues:

Some tests fail
Testing framework doesn’t exit

Steps to reproduce

Follow the guide as far as yarn test.

Expected result

Tests pass, and the testing program exits.

Actual result

Two tests fail, then the testing program keeps counting up forever, not exiting.

Failing tests are:

GraphQL field extensions - allows creating a cutom field extension
GraphQL field extensions - allows creating a custom field extension (used in type builder)

I ran a git bisect. In earlier commits (not sure the threshold for this) the same tests fail but the tester exits, but not before outputting this message:

console.log packages/gatsby-source-wordpress/src/http-exception-handler.js:11
    The request failed with error code "undefined"

Further back in history the tests pass.

The bisect led me to the first commit where the tests fail: 4f9c790b1c1472ed6fb0e020a172766730871e5d

This commit adds the tests and, judging by the commit message, also the code which is being tested.

Environment

  System:
    OS: Linux 4.15 Ubuntu 18.04.2 LTS (Bionic Beaver)
    CPU: (4) x64 Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
    Shell: 5.4.2 - /bin/zsh
  Binaries:
    Node: 10.16.0 - /usr/bin/node
    Yarn: 1.17.3 - /usr/bin/yarn
    npm: 6.9.0 - /usr/bin/npm
  Languages:
    Python: 2.7.15+ - /usr/bin/python
  Browsers:
    Firefox: 68.0
  npmGlobalPackages:
    gatsby-cli: 2

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 1
Comments: 16 (16 by maintainers)

Most upvoted comments

@tremby test.todo is a feature introduced in Jest v24 (I think) and it’s to remember tests that need yet to be written. it can also mean that it has not been decided how some behavior they are supposed to be testing should be implemented. I’m personally more worried by 78 skipped tests

stefanprobst on Jul 26, 2019

@sidharthachatterjee would be cool to also merge #15895 thanks!

stefanprobst on Jul 22, 2019

This was resolved in https://github.com/gatsbyjs/gatsby/pull/15974 and https://github.com/gatsbyjs/gatsby/pull/15975

sidharthachatterjee on Jul 22, 2019

The failed test’s diff


    Difference:

    - Expected
    + Received

      Object {
        "test": Object {
    -     "withDefaultArgs": "Cheers!",
    -     "withQueryArg": ":cake:",
    -     "withoutArgs": "Happy birthday!",
    +     "withDefaultArgs": "2019-09-26",
    +     "withQueryArg": "2019-09-26",
    +     "withoutArgs": "2019-09-26",
        },
      }

Sorta seems like some external data structure has changed. Is this test calling out to a third-party source over the network? If something there changed some time in the last few weeks that’d explain the tests suddenly failing even though the Gatsby code hasn’t changed.

[edit] oh… but you said it’s still passing for you locally. :\

tremby on Jul 19, 2019

Having same issue. Are tests being checked before marking a release? Or maybe something missing on our local environments that isn’t noted in the set up docs.

missmatsuko on Jul 19, 2019