powertools-lambda-python: `child=True` is not creating a logger?

Agreed, given the ability to override location formatting, documentation updates would be sufficient. Ability to suppress keys would be a nice-to-have, but I don’t have a strong opinion there.

A RFC it is, then - if either of you could create I’d be happy to consider it.

I want to avoid a situation where we add too many flags or break existing customers searching mechanisms (saved queries).

Instead, we should provide an option for Power users to override Log records as they see fit without imposing this to less experienced customers, as Python Logging is a complex beast

@heitorlessa looks good

@heitorlessa looks good to me! thanks!

Yes, you can override “location”, “timestamp”, “level”, and “datefmt” - I just haven’t documented them.

If you’re okay with it being this way, then we just need:

1. RFC for suppressing keys, and ordering them (message first, etc)
2. Document this

I’ll check the responses tomorrow

Ok, nice, so overriding location already works:

log = aws_lambda_powertools.Logger(
    service="foo",
    stream=sys.stderr,
    location="%(name)s.%(funcName)s:%(lineno)d",
    timestamp="%(asctime)s.%(msecs)03dZ",
    datefmt="%Y-%m-%dT%H:%M:%S",
)

$ python foo.py
{"timestamp": "2020-08-27T12:54:32.516Z", "level": "INFO", "location": "foo.<module>:14", "service": "foo", "sampling_rate": 0.0, "message": "hallelujah"}
{"timestamp": "2020-08-27T12:54:32.517Z", "level": "INFO", "location": "foo.bar.bar:10", "service": "foo", "sampling_rate": 0.0, "message": "doublestuff"}

Maybe we can allow for a custom pattern for this. As in some cases you don’t want the location field to be too long.

That would be fine. Coming from python logging, we’re used to setting the logformat with our custom preferences.

@michaelbrewer I think it might be easier to have an option to override log record attributes.

When we started, we heard from beta customers the opposite, as the module wasn’t always valuable.

Gotcha. Yeah, the function name is definitely not enough info. Is there some way we can set location to something like package.module.function:line?

@heitorlessa personally I would prefer if “location” included the module it was benign logged from.

This would be similar to other logging libraries.

DEBUG - [method_name] package_name -  message

@heitorlessa Ok, perhaps I’m just not using this library correctly then. All the examples assume service, where python loggers are more typically using the logger name in the record. So I just assumed service === name for the purposes of this library.

I suppose I don’t particularly care about the service name for our use case. I do care about specifically where in the module the message was generated. It looks like this module is using the location for that purpose, but that is just bar instead of the logger name (foo.bar).

$ cat bar.py
import aws_lambda_powertools

log = aws_lambda_powertools.Logger(service="foo", child=True)

def bar():
    log.info('doublestuff')

$ python foo.py
{"timestamp": "2020-08-27 12:02:10,669", "level": "INFO", "location": "<module>:6", "service": "foo", "sampling_rate": 0.0, "message": "hallelujah"}
{"timestamp": "2020-08-27 12:02:10,669", "level": "INFO", "location": "bar:10", "service": "foo", "sampling_rate": 0.0, "message": "doublestuff"}

Hey Loren - Thanks for raising this issue.

If you set service=“foo” in your child in the first example it should work.

We need to make the docs clearer here, so please allow me to explain what’s going on (I’m on my phone so it’ll be a longer response than usual)

Service is what defines what the function is responsible for, or part of (e.g payment service).

For Logger, the service is the logging key customers will use to search log operations for one or more functions, for example - “Search for all error, or messages like X, where service is payment”.

That said, customers can set the service using explicit arguments, but typically use as an environment variable since it’s used across all utilities and to avoid repetition - POWERTOOLS_SERVICE_NAME

With that hopefully explained, let’s get to inheritance.

Python Logging hierarchy happens at the dot notation (service, service.child, service.child_2). For that to work seamlessly, we name all parent Loggers, no child param, with the value defined in the service param.

For child Loggers we introspect the name of your module where Logger(child=True) is called, and name your Logger as “service.filename”.

Once both Loggers are registered, and Python Logging recognises the dot notation, it fixes up the hierarchy and event propagation.

Why the first sample didn’t work

Python Logger doesn’t recognise as a parent + child loggers because their names are different, since service param value is different.

• Parent logger is named “foo”
• Child logger is named “service_undefined”

If you were to use the same service name in the Child logger that should work, as that will then be “foo.bar”, where bar comes from bar.py

Why the service key isn’t foo.bar

That’s because of consistency. The service/application is still the same despite having multiple Loggers.

Sorry for the long explanation but hope that clarifies it - Lemme know otherwise 😃

Thanks