tfsec: bug: tfsec --exclude-downloaded-modules doesn't work for submodules in external modules (since 1.16)

Describe the bug

Since 1.16 the --exclude-downloaded-modules argument doesn’t work anymore on submodules in external modules.

To Reproduce

> tfsec-test $tree
.
├── external-module
│   ├── sns.tf
│   └── submodule
│       └── aws_sns_topic.tf
└── root
    └── main.tf

3 directories, 3 files

> tfsec-test $cat root/main.tf
module "test" {
    source = "git::/mnt/c/work/Bitbucket/tfsec-test/external-module"
}

> tfsec-test $cat external-module/sns.tf
module "submodule" {
    source = "./submodule"
}

> tfsec-test $cat external-module/submodule/aws_sns_topic.tf 
resource "aws_sns_topic" "this" {
  name = "test"
}


Expected behavior

The issues found in the external module are excluded.

Screenshots/Output

> tfsec-test $tfsec --version
v1.18.0
> tfsec-test $tfsec root/ --exclude-downloaded-modules  --concise-output

Result #1 HIGH Topic does not have encryption enabled. 
─────────────────────────────────────────────────────────────────
 .terraform/modules/test/submodule/aws_sns_topic.tf Lines 1-3
───────┬─────────────────────────────────────────────────────────
    1  │ resource "aws_sns_topic" "this" {
    2  │   name = "test"
    3  │ }
───────┴─────────────────────────────────────────────────────────
          ID aws-sns-enable-topic-encryption
      Impact The SNS topic messages could be read if compromised
  Resolution Turn on SNS Topic encryption

  More Information
  - https://aquasecurity.github.io/tfsec/v1.18.0/checks/aws/sns/enable-topic-encryption/
  - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic#example-with-server-side-encryption-sse
─────────────────────────────────────────────────────────────────

System Info

  • tfsec version: v1.18.0
  • terraform version: v0.14.0
  • OS: 20.04.4 LTS (Focal Fossa) (WSL2)

Example Code

See above

Additional context

TFsec tests are passing with versions below 1.16:

> tfsec-test $tfsec --version
v1.15.4
> tfsec-test $tfsec root/ --exclude-downloaded-modules --concise-output

No problems detected!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 3
  • Comments: 18 (6 by maintainers)

Commits related to this issue

Most upvoted comments

@liamg I was at a business trip last week. I’ll have a look this week! Thank you!

Hi @ragchuck - I think this is actually working as intended.

If you point tfsec at the root directory in your example, the --exclude-downloaded-modules will work as you expected - I’ve just verified this on Windows.

image

Unless you point tfsec at a specific directory directly containing *.tf files, tfsec has to figure out which directories are root modules - is does this by recursing down until it finds directories containing *.tf files. Once it finds them, it won’t recurse further into those directories.

In your example above, it finds two directories to treat as project roots: external-module and root. That means external-module is being parsed as a local root module, regardless of the --exclude-downloaded-modules flag.

If you point tfsec exclusively at the root directory, it will only parse that directory as a root module, and the external-module directory can only be loaded as an external module, meaning the flag will be respected.

@liamg sure! as brew doesn’t provide 1.25.1 yet I used the master branch (–HEAD) tfsec . --concise-output --debug

12:26.851361900 system.info                      APP       tfsec
12:26.851429300 system.info                      VERSION   vHEAD-a87d9f8
12:26.851434500 system.info                      OS        linux
12:26.851437100 system.info                      ARCH      amd64
12:26.851439600 system.info                      GOVERSION go1.18.3
12:26.851468500 system.info                      GOROOT    /home/linuxbrew/.linuxbrew/Cellar/go/1.18.3/libexec
12:26.851499400 system.info                      CGO       false
12:26.851509400 system.info                      CPUCOUNT  8
12:26.851539100 system.info                      MAXPROCS  8
12:26.851545100 system.info                      WORKDIR   /mnt/c/work/Bitbucket/tfsec-test
12:26.851548600 system.info                      UID       1000
12:26.851575200 system.info                      EUID      1000
12:26.851580100 system.info                      DOCKER    false
12:26.851582700 system.info                      CI        false
12:26.851608200 system.info                      HOSTNAME  LMUC903681
12:26.851639300 system.info                      TEMP      /tmp
12:26.851648200 system.info                      PATHSEP   /
12:26.851653200 system.info                      CMD       tfsec . --concise-output --debug
12:26.851664800 cmd                              Command args=[]string{"."}
12:26.853459800 cmd                              Determined path dir=/mnt/c/work/Bitbucket/tfsec-test
12:26.855988100 cmd                              Determined path root=/
12:26.856029600 cmd                              Determined path rel=mnt/c/work/Bitbucket/tfsec-test
12:26.867906400 terraform.scanner                Scanning [&{/ /}] at 'mnt/c/work/Bitbucket/tfsec-test'...
12:26.884922200 terraform.scanner.rego           Loaded 4 embedded libraries.
12:26.914097900 terraform.scanner.rego           Loaded 72 embedded policies.
12:26.985921600 terraform.scanner                Scanning root module 'mnt/c/work/Bitbucket/tfsec-test/external-module'...
12:26.985989900 terraform.parser.<root>          Setting project/module root to 'mnt/c/work/Bitbucket/tfsec-test/external-module'
12:26.985995500 terraform.parser.<root>          Parsing FS from 'mnt/c/work/Bitbucket/tfsec-test/external-module'
12:26.990089700 terraform.parser.<root>          Parsing 'mnt/c/work/Bitbucket/tfsec-test/external-module/sns.tf'...
12:26.995672500 terraform.parser.<root>          Added file mnt/c/work/Bitbucket/tfsec-test/external-module/sns.tf.
12:26.996824900 terraform.parser.<root>          Evaluating module...
12:26.996986700 terraform.parser.<root>          Read 1 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
12:26.997054700 terraform.parser.<root>          Added 0 variables from tfvars.
12:26.999338200 terraform.parser.<root>          Error loading module metadata: open //mnt/c/work/Bitbucket/tfsec-test/external-module/.terraform/modules/modules.json: no such file or directory.
12:27.001885200 terraform.parser.<root>          Working directory for module evaluation is '/mnt/c/work/Bitbucket/tfsec-test'
12:27.002073400 terraform.parser.<root>.evaluator Filesystem key is 'f9088d3ed2db9899500f703a07bb505300c2b5cbc122ac4365ca04af35422e64'
12:27.002137300 terraform.parser.<root>.evaluator Starting module evaluation...
12:27.002222600 terraform.parser.<root>.evaluator Starting submodule evaluation...
12:27.002302400 terraform.parser.<root>.evaluator locating non-initialised module './submodule'...
12:27.002313800 terraform.parser.<root>.evaluator.resolver Resolving module 'module.submodule' with source: './submodule'...
12:27.002336700 terraform.parser.<root>.evaluator.resolver Trying to resolve: 9cc0bfafab57e9675997c85141f2eca6
12:27.005613800 terraform.parser.<root>.evaluator.resolver Module 'module.submodule' resolved locally to mnt/c/work/Bitbucket/tfsec-test/external-module/submodule
12:27.005671600 terraform.parser.<root>.evaluator.resolver Module path is mnt/c/work/Bitbucket/tfsec-test/external-module/submodule
12:27.005694900 terraform.parser.<root>.evaluator Module 'module.submodule' resolved to path 'mnt/c/work/Bitbucket/tfsec-test/external-module/submodule' in filesystem '&{/ /}' with prefix ''
12:27.019197500 terraform.parser.<root>.evaluator Loaded module 'submodule' from 'mnt/c/work/Bitbucket/tfsec-test/external-module/submodule'.
12:27.022091100 terraform.parser.<root>.evaluator Finished processing 1 submodule(s).
12:27.022151100 terraform.parser.<root>.evaluator Starting post-submodule evaluation...
12:27.022183200 terraform.parser.<root>.evaluator Module evaluation complete.
12:27.022213500 terraform.parser.<root>          Finished parsing module 'root'.
12:27.022240900 terraform.executor               Adapting modules...
12:27.022397800 terraform.executor               Adapted 2 module(s) into defsec state data.
12:27.022427100 terraform.executor               Using max routines of 7
12:27.022431900 terraform.executor               Applying state modifier functions...
12:27.022434700 terraform.executor               Initialised 335 rule(s).
12:27.022437100 terraform.executor               Created pool with 7 worker(s) to apply rules.
12:27.023064700 terraform.scanner.rego           Scanning 1 inputs...
12:27.060829800 terraform.executor               Finished applying rules.
12:27.060872400 terraform.executor               Applying ignores...
12:27.060912800 terraform.scanner                Scanning root module 'mnt/c/work/Bitbucket/tfsec-test/root'...
12:27.060950000 terraform.parser.<root>          Setting project/module root to 'mnt/c/work/Bitbucket/tfsec-test/root'
12:27.060955400 terraform.parser.<root>          Parsing FS from 'mnt/c/work/Bitbucket/tfsec-test/root'
12:27.065212000 terraform.parser.<root>          Parsing 'mnt/c/work/Bitbucket/tfsec-test/root/main.tf'...
12:27.069553300 terraform.parser.<root>          Added file mnt/c/work/Bitbucket/tfsec-test/root/main.tf.
12:27.070232000 terraform.parser.<root>          Evaluating module...
12:27.070362300 terraform.parser.<root>          Read 1 block(s) and 0 ignore(s) for module 'root' (1 file[s])...
12:27.070410700 terraform.parser.<root>          Added 0 variables from tfvars.
12:27.074824300 terraform.parser.<root>          Loaded module metadata for 3 module(s) from 'mnt/c/work/Bitbucket/tfsec-test/root/.terraform/modules/modules.json'.
12:27.077273700 terraform.parser.<root>          Working directory for module evaluation is '/mnt/c/work/Bitbucket/tfsec-test'
12:27.077664300 terraform.parser.<root>.evaluator Filesystem key is 'f9088d3ed2db9899500f703a07bb505300c2b5cbc122ac4365ca04af35422e64'
12:27.077794800 terraform.parser.<root>.evaluator Starting module evaluation...
12:27.078114700 terraform.parser.<root>.evaluator Starting submodule evaluation...
12:27.078265200 terraform.parser.<root>.evaluator Module 'module.test' resolved to path 'mnt/c/work/Bitbucket/tfsec-test/root/.terraform/modules/test' in filesystem '&{/ /}' using modules.json
12:27.090611300 terraform.parser.<root>.evaluator found module 'git::/mnt/c/work/Bitbucket/tfsec-test/external-module' in .terraform/modules
12:27.090677400 terraform.parser.<root>.evaluator Loaded module 'test' from 'mnt/c/work/Bitbucket/tfsec-test/root/.terraform/modules/test'.
12:27.115121500 terraform.parser.<root>.evaluator Finished processing 2 submodule(s).
12:27.115162500 terraform.parser.<root>.evaluator Starting post-submodule evaluation...
12:27.115209900 terraform.parser.<root>.evaluator Module evaluation complete.
12:27.115254900 terraform.parser.<root>          Finished parsing module 'root'.
12:27.115280000 terraform.executor               Adapting modules...
12:27.115371000 terraform.executor               Adapted 3 module(s) into defsec state data.
12:27.115399800 terraform.executor               Using max routines of 7
12:27.115424400 terraform.executor               Applying state modifier functions...
12:27.115429400 terraform.executor               Initialised 335 rule(s).
12:27.115431700 terraform.executor               Created pool with 7 worker(s) to apply rules.
12:27.115894100 terraform.scanner.rego           Scanning 1 inputs...
12:27.152131200 terraform.executor               Finished applying rules.
12:27.152201500 terraform.executor               Applying ignores...
12:27.152259400 cmd                              Exit code based on results: 1

Result #1 HIGH Topic does not have encryption enabled. 
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────  external-module/submodule/aws_sns_topic.tf:1-3
   via external-module/sns.tf:1-3 (module.submodule)
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────          ID aws-sns-enable-topic-encryption
      Impact The SNS topic messages could be read if compromised
  Resolution Turn on SNS Topic encryption

  More Information
  - https://aquasecurity.github.io/tfsec/vHEAD-a87d9f8/checks/aws/sns/enable-topic-encryption/
  - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic#example-with-server-side-encryption-sse
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Result #2 HIGH Topic does not have encryption enabled.
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────  root/.terraform/modules/test/submodule/aws_sns_topic.tf:1-3
   via git::/mnt/c/work/Bitbucket/tfsec-test/external-module/mnt/c/work/Bitbucket/tfsec-test/root/.terraform/modules/test/sns.tf:1-3 (module.submodule)
    via root/main.tf:1-3 (module.test)
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────          ID aws-sns-enable-topic-encryption
      Impact The SNS topic messages could be read if compromised
  Resolution Turn on SNS Topic encryption

  More Information
  - https://aquasecurity.github.io/tfsec/vHEAD-a87d9f8/checks/aws/sns/enable-topic-encryption/
  - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic#example-with-server-side-encryption-sse
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

  2 passed, 2 potential problem(s) detected.

I wonder if this is a Windows specific issue as I can’t reproduce it here. I’ll try to set this up on Windows and see if the behaviour is different.