opa: opa eval cant parse json file when it used in GitHub Actions. But locally it works.

Short description

I evaluate OPA policy to find the Azure resources without the required tags. Locally it works and outputs non-compliant resources. But when I try to evaluate the policy in the GitHub Actions pipeline - the output is empty. I rechecked everything: local and ‘in pipeline’ configs, versions are the same, all files exist, permissions, etc. In debug mode, I see that “in pipeline” OPA replaces input data with a simple “command” string.

Config: Local and “in pipeline” OPA version - 0.48.0 Local machine - MacOS Ventura Pipeline runner image based on - Ubuntu 22.04

OPA policy:

package policy

mandatory_tags[msg] {
  check_tags(changes[c])
  #msg := sprintf("fail: %v is missing required tags.", [changes[c].address])
	msg := sprintf("%v", [changes[c].address])
}

check_tags(resource) {
 # read the tags based on the resource type
 tags = read_tags(resource)
 # check for the tag enforcement
 match(resource.change.after)
}

read_tags(resource) = tags {
 tags = resource.change.after.tags
}

match(i) {
	not i.tags
}

match(i) {
	not i.tags.AgileTeam
}

match(i) {
	not i.tags.Contact
}

match(i) {
	not i.tags.Environment
}

match(i) {
	not i.tags.Repository
}

# get the changed resources
changes := { c |
	some path, value
	walk(input, [path, value])
	reverse_index(path, 1) == "resource_changes"
	c = value[_]
}

reverse_index(path, idx) = value {
	value := path[count(path) - idx]
}

I run the next commands:

terraform init
terraform plan --out tfplan.binary
terraform show -json tfplan.binary > tfplan.json

opa eval data.policy.all_policies -d ../../.opa/ -i tfplan.json -f pretty

Local output:

[
  "fail: azurerm_nat_gateway.development is missing required tags.",
  "fail: azurerm_public_ip_prefix.development is missing required tags."
]

In pipeline output:

[]

Local debug: Screenshot 2023-02-01 at 16 54 26

In pipeline debug: Screenshot 2023-02-01 at 16 57 38

And, again, all needed files are in place, OPA version is the same. Locally it works but doesn’t work in GitHub Actions Pipeline Screenshot 2023-02-01 at 17 03 14

About this issue

Original URL
State: closed
Created a year ago
Comments: 22 (12 by maintainers)

Most upvoted comments

@anderseknert I do not mean to invalidate @mkurimeti’s experience, but I think I understand why this is happening (and it definitely is still happening for us). If I am correct, it wouldn’t be unique to the self-hosted runners either, but I haven’t tested that yet.

To be brief, this isn’t really an opa bug. It is a “gotcha” that comes from extrapolating the steps at https://www.openpolicyagent.org/docs/latest/terraform for use within GitHub Actions.

To determine this, I inspected our self-hosted runner’s filesystem directly while a Terraform-related opa workflow was running. I noticed something peculiar about the tfplan.json that wasn’t present in local testing and wasn’t showing in the GitHub Actions log itself:

$ cat tfplan.json
[command]/runner/_work/_temp/e16010ee-5ac0-417f-8ffe-7aab4de2e9a0/terraform-bin show -json tfplan
...
::debug::stderr: 
::debug::exitcode: 0

The ... above represents the expected JSON contents, but this “header” and “footer” metadata is also present. Like many others, I was creating the JSON representation of the Terraform plan with a command like:

terraform show -json tfplan > tfplan.json

I believe this extra metadata is being added by the terraform_wrapper that is installed by default when using hashicorp/setup-terraform.

Later, when this file is provided as input to opa, I think it is parsing the first part ([command]) as a YAML array with a single, unquoted string element. This would explain the [ "command" ] output noted earlier in this issue. As an example:

$ echo -e '[command]qwerty\n{}' | opa eval -I -f pretty input
[
  "command"
]

The solution here is to ensure the terraform_wrapper is disabled when terraform show -json ... is executed. Even with the terraform_wrapper enabled, you may be able to usehead and tail to strip the “invisible” GitHub Actions metadata as you write the file.

I am not sure if it is worth including this somewhere in the Open Policy Agent docs. In any case, I hope this helps someone in the future!

nafarlee on Jun 26, 2023

Please do! 👍

anderseknert on Jun 27, 2023

Stellar research! That sounds like a very plausible explanation. While it would be a little unusal to point this out in docs that have nothing to do with GitHub Actions, I think a note might be warranted given how that’s likely a common way to evaluate these plans. Would you like to submit a PR for the docs?

anderseknert on Jun 26, 2023