moon: [bug] Intermittently, cached moon outputs get partially restored

Describe the bug

I’ve been encountering this off and on since adopting moon and had been holding off on reporting it until I could see a pattern on when it happens. Unfortunately I still haven’t but I want to describe the situation and maybe we can figure it out together.

I have a task messaging-api:gen-prisma-client that happens fairly on in my project. The outputs of the task are all sent to a single folder within the source tree of a project. Included in the task outputs are some test files (.js, .d.ts and .json files) and binary files (.node extension) . There are only a couple of inputs for the task and they do not change often which means the output for this task can often be restored from cache.

Intermittently, when this task is executed subsequent tasks that depend upon it will fail because a file that is a .js output of the task has invalid syntax. It appears that when restoring the file from cache only part of a memory buffer is flushed to disk because it looks like the .js file starts halfway through.

I have not noticed if the partially restored file is always a particular part of the file.

Steps to reproduce

I wish I had these ☹️

Expected behavior

The previous directory would be cleared out or the files would be cleanly replaced.

Screenshots

Screenshot 2023-03-21 at 14 52 24

Environment

npx envinfo --system --SDKs --languages --managers --binaries

  System:
    OS: macOS 13.2.1
    CPU: (8) arm64 Apple M1
    Memory: 184.09 MB / 16.00 GB
    Shell: 5.2.15 - /opt/homebrew/bin/bash
  Binaries:
    Node: 16.19.0 - ~/.volta/tools/image/node/16.19.0/bin/node
    Yarn: 3.4.1 - ~/.volta/tools/image/yarn/1.22.11/bin/yarn
    npm: 8.19.2 - ~/.volta/tools/image/npm/8.19.2/bin/npm
    Watchman: 2023.03.13.00 - /opt/homebrew/bin/watchman
  Managers:
    Cargo: 1.68.0 - ~/.cargo/bin/cargo
    Gradle: 8.0.2 - /opt/homebrew/bin/gradle
    Homebrew: 4.0.6 - /opt/homebrew/bin/brew
    Maven: 3.9.1 - /opt/homebrew/bin/mvn
    pip3: 22.3.1 - /opt/homebrew/bin/pip3
    RubyGems: 3.0.3.1 - /usr/bin/gem
  Languages:
    Bash: 5.2.15 - /opt/homebrew/bin/bash
    Go: 1.20.2 - /opt/homebrew/bin/go
    Java: 11.0.17 - /usr/bin/javac
    Perl: 5.30.3 - /usr/bin/perl
    Python3: 3.11.2 - /opt/homebrew/bin/python3
    Ruby: 2.6.10 - /usr/bin/ruby
    Rust: 1.68.0 - /Users/bryanjswift/.cargo/bin/rustc

Additional context

The task I mentioned at the start is defined like this, the :env-compare task does not get cached (it has cache: false in options).

  gen-prisma-client:
    command: 'prisma generate --schema=prisma/schema.prisma'
    deps:
      - 'root:env-compare'
    inputs:
      - '/.env'
      - 'prisma/schema.prisma'
    outputs:
      - 'src/generated/prisma-client'

The output of the task looks like this:

packages/messaging-api/src/generated/prisma-client/
├── index-browser.js
├── index.d.ts
├── index.js
├── libquery_engine-darwin-arm64.dylib.node
├── libquery_engine-rhel-openssl-1.0.x.so.node
├── package.json
├── runtime
│   ├── index-browser.d.ts
│   ├── index-browser.js
│   ├── index.d.ts
│   ├── library.d.ts
│   └── library.js
└── schema.prisma

2 directories, 12 files

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22

Most upvoted comments

Released a patch, let me know if it crops up again.

I’m pretty confident this is resolved. It’s been more than a week since I’ve seen the error reported in this issue. I’m closing it. Thanks @mrobinsn for investigating and @milesj for fixing!

@milesj I think I see the problem here. In the tree differ logic, there is a bit where if the sizes are the same it falls back to a byte level equality check. That involves reading from the source tar, which advances its file pointer, so when you land back into the untar logic after some portion of the source/dest files were not equal, it will only extract data starting from that position. This explains the behavior in the Discord thread where the hydrated files were missing some number of bytes at the beginning.

It might also explain why this happens so rarely and inconsistently. It would only happen when the source and destination both exist, are exactly the same size, but differ in content (but not fully).

Good to know, will dig into.