cypress: Fixture returns outdated/false data

So I asked the team about this. We are indeed caching the reference to the fixture to save memory. This was intended at the time, but is likely not a great solution now.

We likely need to rework this logic, to not use the cache when the file has been changed. We can detect that the file has changed by initially getting the SHA, compare if SHA has changed, otherwise use cache. Alternatively look at modifiedAt time of the file (if works on all platforms)

Workaround:

You can use cy.readFile() instead of fixture as this will not use the cacheing.

@ngokevin

Your database or API responses never change, at all?

So instead of changing the fixtures before running the tests you’re changing them while running the tests?.. Doesn’t that ring a bell? Doesn’t the word “fixutre” mean something fixed, not changing?

The maintainers have flagged it as a bug, I’m not sure why you are filibustering here.

Did you consider that developers look at the issue and think, “Yeah, kind of unexpected. But well, why would they do that? There are more important issues to solve. We’ll get back to it later.” So instead of filibustering, why don’t you put some effort, provide more details, explain why you can’t do it without changing the fixtures, so that everybody (not just you) could understand that yes, indeed, a legitimate case. That would be definitely helpful for people solving the Y problem.

And I suggest you read this: https://xyproblem.info/

P.S. And then they complain that issues stay open for years…

I don’t really see a case for changing fixtures. Fixtures is something prepared in advance and shouldn’t change. If you want to randomize test data, you need faker or something.

Change fixtures to match changes made by API requests? Do you even need fixtures in this case? To answer that question I need more details. What data you have in fixtures? What requests you make?

The bottom line is if you need to change fixtures, that’s most likely a sign you’re doing something wrong. Or so I think for now.

@nephix You seem so confident. Can you possibly back up your words? I can’t reproduce what you’re saying:

The test:

    it('...', () => {
        cy.fixture('1.txt').then(content => {
            cy.log(content);
            cy.writeFile('cypress/fixtures/1.txt', String(Number(content) + 1));
        })
        cy.fixture('1.txt').then(content => {
            cy.log(content);
        });
    });

$ git clone -b cypress-change-fixture https://github.com/x-yuri/issues cypress-change-fixture
$ cd cypress-change-fixture
$ npm i
$ npx cypress open

Then choose 1.spec.js, and you’ll see:

Then press “r” (Run All Tests), and you’ll see:

Which means fixtures don’t persist across sessions. In case they persist for you under some circumstances, please provide exact and easy to follow instructions like I did.

Yeah, this looks like a bug to me - if it’s not, then it’s really unexpected behavior.

I tried using an alias to reference the fixture and also tried using ugly thenables through the Cypress chains to ensure they were being run after another - it always references the original content of the fixture, even though I can see that the content of the fixture has changed.

Reproducible failing test

describe('fixture', () => {
  it('step 1', () => {
    // Create the fixture first
    cy.writeFile('cypress/fixtures/test-temp.json', {
      id: 1,
      name: 'Step 1',
    })

    // Let's see the data, it should be fine
    cy.fixture('test-temp').then((data) => {
      cy.log(data)
      expect(data.name).to.eq('Step 1')
    })

    // Update the fixture again
    cy.writeFile('cypress/fixtures/test-temp.json', {
      id: 2,
      name: 'Step 2',
    })

    cy.fixture('test-temp').then((data) => {
      cy.log(data)
      expect(data.name).to.eq('Step 2')
    })
  })
})

Your database or API responses never change, at all? The maintainers have flagged it as a bug, I’m not sure why you are filibustering here. Anyways, I’ve added my workaround to try to be actually helpful.

Can’t really provide you with a minimal example from my phone

I’m not asking to answer right away, please take your time to make Cypress better.

in my test cases I’ve…

I’ve once again followed your instructions, and still can’t reproduce. Probably because your instructions were not exact. Can you please spend some time and give us a sure way to reproduce your issue? Ideally, create a repository one can clone, npm i, cypress open and see the result.

I already provided instructions above that solve the problem of loaded fixtures being outdated

Let’s make it clear. The issue you’re experiencing is not the issue described in the original post. Because your solution doesn’t solve the original issue. Although both fall under the “outdated fixture data” category. That’s why I’m trying to make you share the exact steps needed to reproduce your issue.

Please see our team’s opinion on this issue here: https://github.com/cypress-io/cypress/issues/4716#issuecomment-518528101

We do think the original issue would be valuable. But also, as mentioned above, there is a workaround and this falls below some other priorities we have at the moment.

We would be open to a PR to change this behavior so that fixture references are not cached.

Thanks for this thread. Wasted a lot of time on this due to the fact that I am not a JS expert 😃 I could clearly see in the console in runner that the contents of the file under fixtures folder is getting updated but my assertion was failing due to cached data of the original content of the fixture. cy.readFile() is a real savior.

I am not sure whether dynamically changing the data of fixture files is a best practice or not - still I hope the bug is fixed soon.

I have multiple tests that interact with the same data object. In our application loading a page requires the same data to pre-populate fields, then we test various front-end functionality. Part of these tests includes what happens when we alter data, save the changes, then reload the page. This is why we need to:

• Know what the new data is when we save the page
• Store this data
• Reload the page and use this new data

Writing this data when it changes simulates our network layer, which hits the update API, then write the new data to a database. Stubbing this improves performance and keeps with the recommended pattern of having one golden-path test that does full end-to-end testing on all network paths, and stubs that simulate our e2e patterns.

For more information:

https://docs.cypress.io/guides/guides/network-requests.html#Stub-Responses

Suggested Use Use for the vast majority of tests Mix and match, typically have one true end-to-end test, and then stub the rest Perfect for JSON APIs

@mwren-mshanken So you use one fixture for many tests and many purposes? That’s not how fixtures are to be used, don’t you agree? The solution is probably to use a different fixture name in different tests.

Speaking of the bigger picture… you do an xhr request, save the body in a fixture, and then stub subsequent requests with the fixture? Is it for performance reasons? Does using fixtures in such a way improve performance a lot?

Since the fixture file never gets updated, this function only works properly the first time it’s called.

By the way, the fixture file is always updated, it’s just that the cy.fixture() always return the first version of a file.

@nephix cy.fixture returns stale data if a fixture changes, what deleting Cypress has to do with it?