nx: `implicitDependencies` dont work when `.gitignore` include the file

Can confirm it is very surprising in any build system for .gitignore to cause a declared dependency to be silently ignored.

Can you all try adding the following to your .nxignore file and see if the issue is resolved?

!tools/executors/**/impl.js

This should negate the .gitignore and everything should work as expected but the file will still be gitignored

Just bumped into this as well with an .env file defining some API URLs which are necessary during build-time webpack bundling.

I managed to get inputs to consider entries within .gitignore by adding a compensating exclusionary entry in .nxignore.

Do note, if the files you want included in the hash sit within a .gitignored directory entry that is not followed by a /*, the files will not be included in the hash. Even if you have an exclusionary entry in .nxignore for the file(s)!

The following configuration ended up working for me:

.gitignore

- someDir
+ someDir/*

.nxignore

!someDir/.env.production.local

project.json

{
  "targets": {
    "build": {
      "inputs": ["{workspaceRoot}/someDir/.env.production.local"]
    }
  }
}

quick answer to myself, nxignore fixed my problem

I have a workaround for this problem until NX decides to fix it:

1. add the following file in tools/cache-file.js

#!/usr/bin/env node

const { existsSync, readFileSync } = require("fs");
const path = require("path");
const chalk = require("chalk");
const { createHash } = require("crypto");

require("yargs").command({
  command: "hash <filepath>",
  describe: `Outputs the hash of the provided file\nUsage: ./${path.basename(
    __filename
  )} hash [filepath]`,
  builder: (yargs) => {
    yargs
      .option("filepath", {
        demandOption: true,
        describe: "Path to the file",
        type: "string",
      })
      .coerce("filepath", (arg) => {
        if (!existsSync(arg)) {
          throw new Error(
            `Provided '${chalk.underline(arg)}' filepath argument doesn't exist`
          );
        } else {
          return readFileSync(arg, "utf8");
        }
      })
      .fail((msg, err, yargs) => {
        console.error(`${chalk.red("[ERROR]")}: ${msg}`);
        console.error(`${yargs.help()}`);
        process.exit(1);
      });
  },
  handler: (argv) => {
    const hashSum = createHash("sha256");
    hashSum.update(argv.filepath);

    console.log(hashSum.digest("hex"));
  },
}).argv;

1. Update your nx.json file as described below:

    "your-task": {
      "inputs": [
        "{workspaceRoot}/.env",
        "{workspaceRoot}/tools/setenv.ts",
        {
          "runtime": "node .\\tools\\cache-file.js hash .env"  // <-- your file that currently gets ignored due to the fact that it's in .gitignore
        }
      ]
    }

Hi, are there any updates in this?

Our applications builds are also depending on files generated by external service, whose are ignored by git.

We were quite fine with manually clearing cache when those file changed, but now we starting to use Nx Cloud and this is something we really struggle with.

All of things mentioned above, we also tried specifying project inputs (https://nx.dev/configuration/projectjson#inputs-&-namedinputs), but without any success.

Or is there anyone who at least figured out a workaround?

Update, adding output of nx report.

  Node : 16.16.0
   OS   : darwin x64
   yarn : 1.22.17
   
   nx : 14.5.6
   @nrwl/angular : 14.5.6
   @nrwl/cypress : 14.5.6
   @nrwl/detox : Not Found
   @nrwl/devkit : 14.5.6
   @nrwl/eslint-plugin-nx : 14.5.6
   @nrwl/express : Not Found
   @nrwl/jest : 14.5.6
   @nrwl/js : 14.5.6
   @nrwl/linter : 14.5.6
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : 14.5.6
   @nrwl/nx-cloud : 14.3.0
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 14.5.6
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : 14.5.6
   @nrwl/web : 14.5.6
   @nrwl/workspace : 14.5.6
   typescript : 4.7.4
   ---------------------------------------
   Local workspace plugins:
   ---------------------------------------
   Community plugins:
         @ionic/angular: 6.2.2
         @ngneat/transloco: 2.23.5
         @nguniversal/express-engine: 12.0.1
         rxjs: 6.6.3
         @nguniversal/builders: 12.0.0
         @nxext/capacitor: 14.0.0
         @nxext/ionic-angular: 14.0.0
         @storybook/angular: 6.5.10

I managed to get inputs to consider entries within .gitignore by adding a compensating exclusionary entry in .nxignore.

Do note, if the files you want included in the hash sit within a .gitignored directory entry that is not followed by a /*, the files will not be included in the hash. Even if you have an exclusionary entry in .nxignore for the file(s)!

The following configuration ended up working for me:

.gitignore

- someDir
+ someDir/*

.nxignore

!someDir/.env.production.local

project.json

{
  "targets": {
    "build": {
      "inputs": ["{workspaceRoot}/someDir/.env.production.local"]
    }
  }
}

After some further investigation, it looks like this behaviour isn’t caused by a quirk in NX, but rather with the underlying ignore crate used by the native file hasher.

There’s been a related open issue for a few years: https://github.com/BurntSushi/ripgrep/issues/1050

quick answer to myself, nxignore fixed my problem

@X4V1 Do you mean this solution?

@a777med wrote that this didn’t work for him. Do you have different output?

EDIT Solution with .nxignore file and using inside it e.g. !path/to/implicitDependency (that is git ignored) worked for me!

I think adding the source file instead of the output file will work better for you. If you’re working locally and amending the output file yourself, then you can use --skip-nx-cache to bypass the cache.

Now I use the source file instead of the output file, but the truth is that cache depend on the output file, so there some question when I change the source file but forgot build output: The cache would be broken when run the task. If we build output that we forgot and re-run the task, the cache would be same as before.