falco: Validation Error when using a macro as definition of an overwritten macro.

What happened / Replication Steps:

Given a macro that is defined in the default ruleset and for the purposes of being concrete let’s use Clear Log Activities:

- macro: allowed_clear_log_files
  condition: (never_true)

Now, if you overwrite this macro in falco_rules.local.yaml, for example:

- macro: allowed_clear_log_files
  condition: >
    (container.name = "I am testing something")

When I run my validation script:

falco -c /etc/falco/falco.yml.erb               \
--validate /etc/falco/falco_rules.yml.erb        \                           
--validate /etc/falco/falco_macros.local.yml.erb  \

Everything succeeds 🎉

However, if I overwrite this macro in falco_rules.local.yaml using another macro, for example:

- macro: allowed_clear_log_files
  condition: >
    consider_all_chmods

I get the following error message: Runtime error: Error loading rules: /usr/share/falco/lua/compiler.lua:65: Undefined macro 'consider_all_chmods' used in filter.. Exiting.

(It is important to note that consider_all_chmods is in defined in the default ruleset, so I am not sure why it is failing here.

Anything else we need to know?:

Environment:

Falco version (use falco --version): falco 0.14.0
System info
Cloud provider or hardware configuration:
OS (e.g: cat /etc/os-release):
Kernel (e.g. uname -a):
Install tools (e.g. in kubernetes, rpm, deb, from source):
Others:

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 41 (24 by maintainers)

Most upvoted comments

@kris-nova is going to start looking at this. It would be good to discuss on the upcoming repo planning call to get an idea of how long it might take to fix.

mfdii on Sep 10, 2019

Here’s some background about how the objects are maintained and how it can cause this bug. When the objects are read they’re maintained in an ordered array. The ordered array is necessary to assure that appends are applied in the right order.

I think the problem is that the macro allowed_clear_log_files replaces the instance of allowed_clear_log_files in that array (in this case, third). Since it’s third, the macro reference test_macro doesn’t exist yet.

I think the fix is to be more liberal about using macros defined later in the file when expanding macro references. We’ll have to make sure that semantics about ordering still apply wrt appends and overrides.

mstemm on Jul 11, 2019

Hey @wdoekes Would you like to open a PR? 🤩 Maybe @jasondellaluce is interested too.

leogr on Nov 18, 2021

Hello!

I still experience this issue.

In my falco_rules.local.yaml file, if I define my own macro then try to use it somewhere in the falco_rules.local.yaml file, it does not work. This is a little bit annoying because without that feature, I have to repeat the exact same condition over and over again. Being able to define my own macros would allow me to create cleaner rules. See #504 for an example.

Can someone add a tag that will prevent that bot from closing relevant issues? 😄

Thank you 😃

JPLachance on Aug 3, 2020

We still want to fix this.

mstemm on Sep 10, 2019

For the sake of completeness, this is the generated AST for the last rules that @natalysheinin sent in this comment

click me to see the ast dump

{
  allowed_clear_log_files = {
    ast = {
      type = "Macro",
      value = "never_true"
    },
    used = false
  },
  consider_all_chmods = {
    ast = {
      type = "Macro",
      value = "never_true"
    },
    used = false
  },
  never_true = {
    ast = {
      left = {
        type = "FieldName",
        value = "evt.num"
      },
      operator = "=",
      right = {
        type = "Number",
        value = 0
      },
      type = "BinaryRelOp"
    },
    used = true
  },
  open_write = {
    ast = {
      left = {
        left = {
          left = {
            left = {
              left = {
                type = "FieldName",
                value = "evt.type"
              },
              operator = "=",
              right = {
                type = "BareString",
                value = "open"
              },
              type = "BinaryRelOp"
            },
            operator = "or",
            right = {
              left = {
                type = "FieldName",
                value = "evt.type"
              },
              operator = "=",
              right = {
                type = "BareString",
                value = "openat"
              },
              type = "BinaryRelOp"
            },
            type = "BinaryBoolOp"
          },
          operator = "and",
          right = {
            left = {
              type = "FieldName",
              value = "evt.is_open_write"
            },
            operator = "=",
            right = {
              type = "BareString",
              value = "true"
            },
            type = "BinaryRelOp"
          },
          type = "BinaryBoolOp"
        },
        operator = "and",
        right = {
          left = {
            type = "FieldName",
            value = "fd.typechar"
          },
          operator = "=",
          right = {
            type = "String",
            value = "f"
          },
          type = "BinaryRelOp"
        },
        type = "BinaryBoolOp"
      },
      operator = "and",
      right = {
        left = {
          type = "FieldName",
          value = "fd.num"
        },
        operator = ">=",
        right = {
          type = "Number",
          value = 0
        },
        type = "BinaryRelOp"
      },
      type = "BinaryBoolOp"
    },
    used = false
  }
}

The test_macro is clearly missing, proving that it was not considered.

fntlnz on Jul 5, 2019

Do you have any recommendations for how to approach solving this from the user-end for now?

The workaround I’d use right now would be putting everything in one file (making sure that test_macro is above allowed_clear_log_files macro).

Clearly the edits to the creation and the processing of the macros tree needs to fix also this case.

leodido on Jul 5, 2019

Yes @natalysheinin, unfortunately this is due to the fact that test_macro macro does not exist in the default rules file (ie., falco_rules.yaml).

And this is very correlated to the original problem you reported. Also this is happening because all the processing (particularly the creation of the macros tree) that rule_loader.lua does are based solely on the default rules file.

leodido on Jul 5, 2019