marshmallow: "unknown" option should apply to nested fields so that "exclude" works

The fact that exclude takes dotted notation for nested fields, but unknown does not apply to nested fields, means that exclude doesn’t work for nested fields in some common use cases unless the nested schema(s) have the desired Meta.unknown setting, and then they are not runtime-flexible, which is the great benefit of Schema and load taking exclude. Somewhat new to marshmallow so maybe there’s a way to accomplish this (setting exclude on nested fields at load time) in a different way?

Happy to contribute a PR if you guys think this is a good idea.

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 1
Comments: 15 (11 by maintainers)

Commits related to this issue

Add changelog and docs for propagate_unknown The changelog entry, including credit to prior related work, covers the closed issues and describes how `propagate_unknown` is expected to behave. Inline... — committed to sirosen/marshmallow by sirosen 4 years ago
Add changelog and docs for propagate_unknown The changelog entry, including credit to prior related work, covers the closed issues and describes how `propagate_unknown` is expected to behave. Inline... — committed to sirosen/marshmallow by sirosen 4 years ago
Add changelog and docs for propagate_unknown The changelog entry, including credit to prior related work, covers the closed issues and describes how `propagate_unknown` is expected to behave. Inline... — committed to sirosen/marshmallow by sirosen 4 years ago

Most upvoted comments

any news here?

mahenzon on Oct 12, 2020

For the people looking how to solve the issue, this helps me:

“user”: fields.Nested(
{“id”: fields.Int(required=True),
“login”: fields.Str(required=True), “url”: fields.Str()
},
required=True,
unknown=EXCLUDE),

Adding unknown=EXCLUDE inside the fields.Nested to make sure that the Nested field ignore other data coming from the request.

caio-vinicius on Apr 4, 2022

Looking at this issue from the perspective of webargs, I see two (related) issues with #1575. I think I have a good idea of how to resolve them, but want to see if people like the results before I try to write it.

If unknown propagates automatically in any context (as in #1575), it will be backwards incompatible
If unknown behaves inconsistently between Meta, schema instantiation, and load, it will cause confusion

And then, as an ancillary concern, if someone is writing a library on top of marshmallow (👋 hi guys!), they’ll want to be able to control this or offer control to their users.

Maybe backwards incompatibility isn’t an issue in this case – if it’s considered a bug that’s being fixed – but I think it will be a problem for someone.

So my idea:

add a new parameter in all of the same places as unknown, propagate_unknown
propagate_unknown can be True or False (by default, for backwards compat) and can be removed (True everywhere) in marshmallow v4
if propagate_unknown is True, behave as in #1575 when unknown is passed to load, and if False, behave as today

I would take #1575 as a basis for this work. It looks solid, has nice tests, and credit where it’s due – I just don’t quite agree with the behavior.

The downsides are that we have a new attribute to control unknown, which might feel messy, and that we don’t get the desired behavior by default. To me, those are worthwhile in the first case and a necessary evil in the second.

You can do it all with just one attribute, but I promise that it's worse.

This is just for fun. Don’t take this suggestion seriously. Please. 😄

We could change the constants a bit…

RAISE = 1
INCLUDE = 2
EXCLUDE = 4
PROPAGATE = 8
...
schema.load(data, unknown=EXCLUDE|PROPAGATE)

but then we’re just asking for trouble because you have to check for INCLUDE|EXCLUDE, which makes no sense.

sirosen on May 15, 2020

I copied from Meta.exclude, but linked to Schema(exclude). The constructor parameter definition is clear and matches the implementation. The Meta.exclude definition should probably be fixed to match that.

deckar01 on Jan 15, 2020