go: errors: add support for wrapping multiple errors
For the most recent version of this proposal, see: https://github.com/golang/go/issues/53435#issuecomment-1191752789 below.
This is a variation on the rejected proposal #47811 (and perhaps that proposal should just be reopened), and an expansion on a comment in it.
Background
Since Go 1.13, an error may wrap another by providing an Unwrap
method returning the wrapped error. The errors.Is
and errors.As
functions operate on chains of wrapped errors.
A common request is for a way to combine a list of errors into a single error.
Proposal
An error wraps multiple errors if its type has the method
Unwrap() []error
Reusing the name Unwrap
avoids ambiguity with the existing singular Unwrap method. Returning a 0-length list from Unwrap
means the error doesn’t wrap anything. Callers must not modify the list returned by Unwrap
. The list returned by Unwrap
must not contain any nil
errors.
We replace the term “error chain” with “error tree”.
The errors.Is
and errors.As
functions are updated to unwrap multiple errors. Is
reports a match if any error in the tree matches. As
finds the first matching error in a preorder traversal of the tree.
The errors.Join
function provides a simple implementation of a multierr. It does not flatten errors.
// Join returns an error that wraps the given errors.
// Any nil error values are discarded.
// The error formats as the text of the given errors, separated by sep.
// Join returns nil if errs contains no non-nil values.
func Join(sep string, errs ...error) error
The fmt.Errorf
function permits multiple instances of the %w
formatting verb.
The errors.Split
function retrieves the original errors from a combined error.
// Split returns the result of calling the Unwrap method on err,
// if err's type contains an Unwrap method returning []error.
// Otherwise, Split returns nil.
func Split(err error) []error
The errors.Unwrap
function is unaffected: It returns nil
when called on an error with an Unwrap() []error
method.
Questions
Prior proposals have been declined on the grounds that this functionality can be implemented outside the standard library, and there was no good single answer to several important questions.
Why should this be in the standard library?
This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in errors.Is
and errors.As
. Existing combining errors operate by providing Is
and As
methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways. This is best handled in errors.Is
and errors.As
, for the same reason those functions handle singular unwrapping.
In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.
How are multiple errors formatted?
A principle of the errors
package is that error formatting is up to the user. This proposal upholds that principle: The errors.Join
function combines error text with a user-provided separator, and fmt.Errorf
wraps multiple errors in a user-defined layout. If users have other formatting requirements, they can still create their own error implementations.
How do Is
and As
interact with combined errors?
Every major multierror package that I looked at (see “Prior art” below) implements the same behavior for Is
and As
: Is
reports true if any error in the combined error matches, and As
returns the first matching error. This proposal follows common practice.
Does creating a combined error flatten combined errors in the input?
The errors.Join
function does not flatten errors. This is simple and comprehensible. Third-party packages can easily provide flattening if desired.
Should Split
unwrap errors that wrap a single error?
The errors.Split
function could call the single-wrapping Unwrap() error
method when present, converting a non-nil result into a single-element slice. This would allow traversing an error tree with only calls to Split
.
This might allow for a small improvement in the convenience of code which manually traverses an error tree, but it is rare for programs to manually traverse error chains today. Keeping Split
as the inverse of Join
is simpler.
Why does the name of the Split
function not match the Unwrap
method it calls?
Giving the single- and multiple-error wrapping methods the same name neatly avoids any questions of how to handle errors that implement both.
Split
is a natural name for the function that undoes a Join
.
While we could call the method Split
, or the function UnwrapMultiple
, or some variation on these options, the benefits of the above points outweigh the value in aligning the method name with the function name.
Prior art
There have been several previous proposals to add some form of combining error, including:
https://go.dev/issue/47811: add Errors as a standard way to represent multiple errors as a single error https://go.dev/issue/48831: add NewJoinedErrors https://go.dev/issue/20984: composite errors https://go.dev/issue/52607: add With(err, other error) error fmt.Errorf(“%w: %w”, err1, err2) is largely equivalent to With(err1, err2).
Credit to @jimmyfrasche for suggesting the method name Unwrap
.
There are many implementations of combining errors in the world, including:
https://pkg.go.dev/github.com/hashicorp/go-multierror (8720 imports) https://pkg.go.dev/go.uber.org/multierr (1513 imports) https://pkg.go.dev/tailscale.com/util/multierr (2 imports)
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 110
- Comments: 93 (69 by maintainers)
Links to this issue
Commits related to this issue
- go/analysis/passes/stdmethods: recognize Unwrap() []error Recognize the new multiple-error-wrapping method signature. For golang/go#53435 Change-Id: Ifba25746323d036d1e6d3e6d3c34cd6ce904b60a Review... — committed to golang/tools by neild 2 years ago
- go/analysis/passes/printf: permit multiple %w format verbs For golang/go#53435 Change-Id: Icdc664585fbcf7ac07ef92df8b43b20c1d7733e2 Reviewed-on: https://go-review.googlesource.com/c/tools/+/432576 R... — committed to golang/tools by neild 2 years ago
- cmd: update vendored golang.org/x/tools for multiple error wrapping Updates vet to permit errors with an Unwrap method returning []error and multiple %w verbs in fmt.Errorf. For #53435. Change-Id: ... — committed to golang/go by neild 2 years ago
- errors, fmt: add support for wrapping multiple errors An error which implements an "Unwrap() []error" method wraps all the non-nil errors in the returned []error. We replace the concept of the "erro... — committed to golang/go by neild 2 years ago
- doc/go1.20: add a release note for multiple error wrapping For #53435. Change-Id: I894bd645b0b61d7dd5f3aae7d1ea7b8a12f31dd8 Reviewed-on: https://go-review.googlesource.com/c/go/+/450376 Run-TryBot: ... — committed to golang/go by neild a year ago
- util/multierr: add Error.Unwrap and Range Have Error implement the new interface for Unwrap coming in Go 1.20, where Unwrap can either return an error or an []error. See golang/go#53435. Errors in G... — committed to tailscale/tailscale by dsnet a year ago
- util/multierr: add Range Errors in Go are no longer viewed as a linear chain, but a tree. See golang/go#53435. Add a Range function that iterates through an error in a pre-order, depth-first order. ... — committed to tailscale/tailscale by dsnet a year ago
- util/multierr: add Range Errors in Go are no longer viewed as a linear chain, but a tree. See golang/go#53435. Add a Range function that iterates through an error in a pre-order, depth-first order. ... — committed to tailscale/tailscale by dsnet a year ago
- util/multierr: add Range (#6643) Errors in Go are no longer viewed as a linear chain, but a tree. See golang/go#53435. Add a Range function that iterates through an error in a pre-order, depth-f... — committed to tailscale/tailscale by dsnet a year ago
- util/multierr: add Range (#6643) Errors in Go are no longer viewed as a linear chain, but a tree. See golang/go#53435. Add a Range function that iterates through an error in a pre-order, depth-f... — committed to coder/tailscale by dsnet a year ago
- Support Go 1.20 Unwrap() []error Go 1.20 includes native support for wrapping multiple errors. Errors which wrap multiple other errors must implement, Unwrap() []error If an error implements th... — committed to abhinav/multierr by abhinav a year ago
- Support Go 1.20 Unwrap() []error (#69) Go 1.20 includes native support for wrapping multiple errors. Errors which wrap multiple other errors must implement, Unwrap() []error If an error im... — committed to uber-go/multierr by abhinav a year ago
- errors, fmt: add support for wrapping multiple errors An error which implements an "Unwrap() []error" method wraps all the non-nil errors in the returned []error. We replace the concept of the "erro... — committed to fancybits/go by neild 2 years ago
Multiple error wrapping will be in 1.20.
No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal. — rsc for the proposal review group
Updated proposal, incorporating the following changes:
Split
.Join
.Join
uses"\n"
as the separator.If the slice of errors passed toJoin
contains exactly one non-nil error,Join
returns that error.Fixed specification of tree traversal performed byAs
to inorder, not preorder.Proposal
An error wraps multiple errors if its type has the method
Reusing the name
Unwrap
avoids ambiguity with the existing singular Unwrap method. Returning a 0-length list fromUnwrap
means the error doesn’t wrap anything. Callers must not modify the list returned byUnwrap
. The list returned byUnwrap
must not contain anynil
errors.We replace the term “error chain” with “error tree”.
The
errors.Is
anderrors.As
functions are updated to unwrap multiple errors.Is
reports a match if any error in the tree matches.As
finds the first matching error in ainorderpreorder traversal of the tree.The
errors.Join
function provides a simple implementation of a multierr. It does not flatten errors.The
fmt.Errorf
function permits multiple instances of the%w
formatting verb.The
errors.Unwrap
function is unaffected: It returnsnil
when called on an error with anUnwrap() []error
method.Questions
Prior proposals have been declined on the grounds that this functionality can be implemented outside the standard library, and there was no good single answer to several important questions.
Why should this be in the standard library?
This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in
errors.Is
anderrors.As
. Existing combining errors operate by providingIs
andAs
methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways. This is best handled inerrors.Is
anderrors.As
, for the same reason those functions handle singular unwrapping.In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.
How are multiple errors formatted?
The
errors.Join
function separates the text of the joined errors with newlines. Thefmt.Errorf
wraps multiple errors in a user-defined layout. If users have other formatting requirements, they can still create their own error implementations.How do
Is
andAs
interact with combined errors?Every major multierror package that I looked at (see “Prior art” below) implements the same behavior for
Is
andAs
:Is
reports true if any error in the combined error matches, andAs
returns the first matching error. This proposal follows common practice.Does creating a combined error flatten combined errors in the input?
The
errors.Join
function does not flatten errors. This is simple and comprehensible. Third-party packages can easily provide flattening if desired.Why is there no function to unwrap a multiply-wrapped error?
While existing multierror implementations generally provide a mechanism for extracting the individual errors from a combined error, this functionality seems to be used infrequently in practice.
A package which wishes to provide a list of errors for the caller to iterate through should either return an explicit
[]error
(which the caller mayJoin
into a single error if needed), or a concrete type that wraps the contained errors. For example:Returning a concrete type like
ValidationErrors
has the advantage of permitting iteration over the individual errors even if the multiply-wrapped error has been itself wrapped.How should existing multierror types adopt the new interface?
A goal is for existing multierror implementations to implement the common multiple-unwrap interface.
Most existing multierr implementations include
Is
andAs
methods on their error type which descend into each wrapped error. If such an implementation simply adds anUnwrap() []error
method with no other changes,errors.Is
anderrors.As
will examine the wrapped errors twice: Once when the customIs
orAs
method is called, and a second time when descending into the wrapped errors returned byUnwrap() []error
.To avoid this double-examination, multierr implementations which add support for
Unwrap() []error
should use build constraints to defineIs
andAs
methods when built with older versions of Go, and anUnwrap() []error
method when built with newer ones.[Edited: Changed inorder back to preorder] [Edited: Corrected typo:
Unwrap() error
->Unwrap() []error
]I grepping in my own dependencies and grabbed all the uses. I ignored the standard library, static analyzers, linters, and then pulled all the uses that were not in tests. Here they are:
x/tools uses errors.Unwrap to get to the innermost error, without regard for the structure of what is above it. I suspect that this could would be better served by being able to evaluate a predicate on every error in the tree.
aws-sdk-go uses errors.Unwrap to recursively evaluate a predicate on every error in the tree, with early stopping.
containerd uses errors.Unwrap in a way that I find confusing and might be buggy. It uses it to peel back exactly one layer of wrapping and then compares this to a particular error using string matching. This is fragile (if another wrapping layer gets introduced) and would probably be better served by a way to check every error in the tree.
containerd uses errors.Unwrap again to peel back exactly one layer of wrapping. It looks like errors.As would be a better fit here; I don’t see any reason why exactly one layer is the right answer here.
jwt uses errors.Unwrap to implement Is. In this case, errors.Unwrap is unnecessary, because e is known to have a concrete type that implements Unwrap; the code could call e.Unwrap directly instead.
I then grabbed two uses in tests at random:
errwrap uses errors.Unwrap specifically to test that it is compatible with how the standard library defines errors.Unwrap. It has no actual use for errors.Unwrap or opinion about its behavior.
pgconn uses errors.Unwrap to test its own error’s compability with package errors. It looks like it would be equally well served by errors.As.
So far, I see no evidence that anyone cares much about the structure of the error tree. It looks to me like they are all: misusing the API (evidence that the API is wrong); using the wrong API (evidence that the API is wrong); using the API pointlessly; or using the API to evaluate a predicate “anywhere along the error chain”.
I was sort of surprised looking at the
errors.Join
documentation that it is not mentioned that the returnederror
implements theUnwrap() []error
method. In the absence of theerrors.Split
function, this method has significantly reduced discoverability. Making it more discoverable would ease the issue of how to format multiple errors — at least in this case — for different use cases because it would become obvious to users that there was an easy path from the returned value oferrors.Join
to a[]error
that they could format as they wish.The addition would not need to be more than “A non-nil error returned by Join implements the Unwrap() []error method.”.
This strikes me as a significant step forward in figuring out the multi-error question. Thank you!
The only part of this that doesn’t seem entirely spot-on is
errors.Split
.What is the use case for
errors.Split
?Figuring
errors.Split
would be used likeerrors.Unwrap
, I grepped through the code sitting around on my laptop and found only a single use oferrors.Unwrap
. It was used to check recursively whether any error in the chain satisfied a predicate. It would be interesting to analyze other uses in a larger corpus.Walking an arbitrary error tree using
errors.Split
anderrors.Unwrap
will be annoying. I wonder whether a callback-basederrors.Walk
that walks the entire error tree would be a better API. There are a lot of details to specify for such an API, but with a bit of care, it could be more powerful and flexible thanerrors.Split
. It could be used to implementerrors.Is
anderrors.As
. And it sidesteps questions about ownership and modification of returned error slices.(The details about
errors.Walk
include things like: Does it call back for all errors, or only leaf errors? Does it use a single callback likeio/fs.WalkDir
or separate pre- and post- callbacks likegolang.org/x/tools/ast/astutil.Apply
? Does it support early stopping?)On the topic of slice copies, a minor question. Does
errors.Join
make a copy of the variadic arg? I’d argue for yes, and ameliorate the pain a bit by adding a small default backing store to the struct. Aliasing bugs are miserable.This has a been a very long conversation, but it has also been focused and productive and fun, and I think we’re starting to converge(?). Thank you.
One small tweak I would suggest to the Join docs:
This avoids pointless wrapping.
…and I just realized I mixed two topics in a single thread, which makes emoji voting hard. Oops. I’ll assume any thumbs up/down are purely for the technical content. 😃
I think the evidence here is that a bulleted list is not the right formatting option for the
Error
method. None of the implementations surveyed do a good job at all of formatting nested multierrs. (Uber comes the closest, by virtue of only applying the bulleted-list formatting to the outermost error.)That leaves a user-provided delimiter (the original Join proposal) or a fixed delimiter.
How about
"\n"
? Semicolon-separated errors get unreadable quickly, and placing each error on a separate line gets most of the benefits of a bulleted list without the problems caused by nested indenting. Tree structure of nested errors gets lost, but@josharian’s last two test cases of nested errors using newlines as a separator:
https://go.dev/play/p/a7gHGrNst9- https://go.dev/play/p/c1Y7GYwEAyj
I’m going to go look, but probably not until next week. Had a few days of meetings, and I’m now exhausted and behind on everything. 😃
This case turns on whether
errj == errj
. If it does, thenIs
reports true.My unwritten assumption has been that a joined error is equal to itself, and that the underlying type of the error returned by
Join
is something like*struct { errs []error }
.errors.Is
doesn’t unwrap the target today. That wouldn’t change. This is important to keep the complexity ofIs
reasonably bounded.So
is false, because the target doesn’t match either
err1
orerr2
.Every multierr implementation I’ve surveyed has defined
errors.Is(multierr, target)
as true iftarget
matches any error inmultierr
. There may be a use case for an exact match, but I haven’t observed it in the wild. I think this argues for this being the generally expected behavior.Musing…one downside to the multi-line w/ bullets formatting is that it doesn’t nest well (without magic).
Consider
errors.Join(errors.Join(errors.New("A"), errors.New("B")), errors.Join(errors.New("C"), errors.New("D")))
. How should this format using e.g. “newlines+asterisk”? IIUC, it’d naively format as:Given the size of these error trees, where even “a dozen” would be huge, I would suggest the solution to iteration is simply to define that the traversal used by the rest of the library will be used to return a []error populated in the traversal order used by the rest of the code, or a simple tree struct. Such a small set of values doesn’t rate having the iteration proposal placed as a blocker in front of it.
It’s rarely used functionality (as observed above for Unwrap in general), on the error path (not the happy path where performance is generally a bigger deal), for a rarely-used use case where Is and As is not enough. Code that is generating millions or billions of complex error trees per second that it then somehow needs to deal with in a complicated manner is too broken to be saved by even zero-performance error tree iteration.
I’m 100% on
Unwrap() []error
Is
/As
to support the newUnwrap
fmt.Errorf
to allow multiple%w
If those were the only things accepted I’d be happy and it would be more than enough to allow external multierrors to interoperate with each other and std.
I do think there 100% does need to be a tree walking API—but it’s too early to decide what that looks like. Once the protocol above is enshrined in
errors
, it’s simple enough to experiment with that outside std.I’m not 100% sold on the name for
Split
but there should probably be such a func in std, regardless of name, to pair withUnwrap
. I don’t think it would be much of an impediment if this weren’t included, however.I’m not really sold on
Join
. I see the utility of including a basic multierror in std, but just gluing the errors together with a sep seems like it could get messy when multierrors get composed. I think, like a tree walking API, it should be left out and experimentation allowed to continue outside std for now.I’ve been thinking about this case, and I now think it’s a mistake.
Join
should consistently return an error with anUnwrap() []error
method.err
” and “a list of errors containing onlyerr
”.That is an interesting point that didn’t occur to me until now, but it also brings to mind what the semantics of
errors.Is
when the target is a multi-error.I realize this may not come up much in most code since the target is often a sentinel error value. I can imagine it possibly coming up in test code checking if returned errors are consistent across an API perhaps.
I suspect this isn’t a big concern, but wanted to mention it in case anyone else has thoughts or concerns about it.
My main argument against Join/Split is that it’s easy to punt that decision to 3rd party implementations to work out the details before anything gets enshrined in the Go 1 compatibility agreement. Defining the interface and how Is/As work is enough to let everything work together and experiment.
I very much like the simplicity of a join function that takes a
[]error
and returns anerror
, as opposed to an append function which adds more errors to an existing one. I think join is the right API for the standard library. Append opens too many questions about tree structure and allocations.The sticky question is what the text of a joined error should be. The nice thing about
Join(sep, errs)
with an explicit separator is that it has no opinion; the user remains under full control of the error text. The problem with it is that existing implementations have a clear preference for bulleted lists of errors, which you can’t do with just a separator. So perhaps we can’t get away from the need to be somewhat opinionated about error text.If we don’t, there is no standard way to go from an unknown number of errors to a single error (or nil).
fmt.Errorf
supporting multiple%w
verbs is awesome, but only supports a fixed number of errors.I think we need some API that goes from an unknown number of errors to a single error, because this is a very common use case. It could be one-shot (take in a slice) or incremental (add errors one at a time).
I don’t have strong feelings about exactly what that API is. I was happy with
Join
and would probably be happy with others as well.I agree that the data seems to indicate that unwrapping one level of
[]error
is rare, and that in the rare cases that it is done the justification is often dubious.Looking back over the above discussion, I think there are three major questions.
Should we provide
Join
?@jimmyfrasche says:
Looking at some existing multierr implementations:
github.com/hashicorp/go-multierror
formats as a*
bulleted list with a header line.go.uber.org/multierr
joins the errors with;
, and formats%+v
as a*
bulleted list with a header line.tailscale.com/util/multierr
formats as a-
bulleted list with a header line.https://go.dev/play/p/zwGWvtR0IHW
It seems as if existing implementations have converged on providing a list with one error per line, as an option if nothing else. This may argue that
Join
is the wrong API here.Should we provide
Split
?@josharian says:
I think that the evidence from calls to
"go.uber.org/multierr".Errors
supports this argument. It feels very strange to me to not provide a simple way to unpack a multierr, but I’m convinced that this API is an invitation to misuse. The rare cases that actually need to unpack one level of a multierr can use a type assertion and call theErrors
method directly.Should we provide a tree walking API?
None of the existing multierr implementations I’ve looked at provide this. If this is useful, we should demonstrate it outside the standard library first.
I’m not so sure that we do.
For uses (testing?) in which you really specifically want to unwrap from an error that has an
Unwrap() []
method, you can use a type assertion. (errors.Unwrap
is less than 10 LOC, and consists entirely of a type assertion.)But I suspect that almost nobody wants specifically to unwrap a multi-error exactly one layer deep. Rather they want to ask the general question “what errors are in this tree?”. So I think that’s the right form of API to add. And I think it’ll get created internally anyway to implement Is and As.
Maybe this proposal should simply omit
Split
?Depth-first. Unwrap to an
[]error
and walk eacherror
in the list in turn. This is what every existing multierr implementation I looked at does.it’s really sad, I have no understanding why you need it. just will complicate the apps a lot and make people use it in a weird way because we are not smart in general.
Change https://go.dev/cl/432898 mentions this issue:
errors, fmt: add support for wrapping multiple errors
Change https://go.dev/cl/432576 mentions this issue:
go/analysis/passes/printf: permit multiple %w format verbs
What is the final version of the proposal though?
@neild (the author) said
Separately, @neild would you consider capturing all the weird cases like
errj := errors.Join(err2, err3); errors.Is(errors.Join(err1, errj), errj)==?
discussed throughout in the comments as part of the proposal itself? It might make it easier to grok from a single place.Various discussion here over the last two weeks, but I don’t see any objections to accepting this proposal. Do I have that right? Does anyone object to the proposal?
I think Uber’s behavior is because it doesn’t plumb the
%+v
down into contained errors, so everything under the top level formats as;
-separated on a single line.Sampling of existing packages handling formatting of nested multierrs: https://go.dev/play/p/7L2Ydp4-JdZ
I’ve compiled some data on use of splitting multierrors created with
go.uber.org/multierr
back inmto the original list of errors.I started with a list of 1976 modules known to
pkg.go.dev
which importgo.uber.org/multierr
. I populated my local module cache with this set by runninggo get $MOD@latest
for each one. I then usedgrep
to build a list of.go
files in my module cache that imported this package, and filtered this down to only files in the latest version of each module when there were duplicates. There’s probably a more elegant way to do this, but it worked for me.This gave me 5981 unique files that import
go.uber.org/multierr
.Of those, 182 call
multierr.Error
to convert anerror
into a[]error
.The following is a set of links to uses of
multierr.Error
calls, with a few deleted packages removed. Many of these look to be forks of the same original code; I haven’t made any effort to deduplicate these.expand
Looks great and can’t wait to apply it to engineering!
Re @carlmjohnson it’s unclear what we can do except caution people about rollout. Tim’s
var _ = errors.Join
(no need to call it and cause runtime overhead) would be fine for people who want to be extra cautious. I don’t see how to do a vet check with sufficient precision, since it won’t bother the majority of people, and a go.mod in a dependency is allowed to be newer than the toolchain. (Also the older toolchain won’t have the vet check.)It doesn’t sound like there are any other objections, so moving this to likely accept.
I’m sorry, @BenjamenMeyer , I may be misunderstanding this comment. Could you clarify what you mean by the interface being fully implemented?
I also want to state that my last comment is only about the return value of
errors.Join
. My suggestion is that the returned error byerrors.Join
may implement theUnwrap() []error
method but does not have to—depending on how many non-nil errors were provided to it. This is similar to how the error returned byfmt.Errorf
may implementUnwrap() error
,Unwrap() []error
, or neither depending on how many%w
s were present in the template string.I’m suggesting that because the advantage of changing that is not clear to me from the discussion above. @neild suggested three reasons:
(The third one is less a reason to do this, and more a reason why we can do this, so I’ll skip that.)
For the first one, I don’t think we actually get additional safety from making this change. The change will not enable, say, unchecked type casts of returned errors because of how the errors are intended to be used. Functions will normally return joined errors to their callers, with some code paths returning plain errors. So callers still need to match on the interface if they want to access individual errors directly. E.g. given,
Callers of
foo()
cannot assume that the returned error implementsUnwrap() []error
because ifos.Open
failed, it’s a plain error.@neild, maybe there’s something more to this point you made. Would you mind elaborating on this?
Perhaps we need a new
errors.Has
orerrors.Contains
to allow for thecontains
version and then specify thaterrors.Is
is theexact
error. More choice of methods may be the solution.Several thoughts:
We should not expose the ability to specify the delimiter to
Join
.Rather, we should choose something reasonable, whether it be
;
or\n
or even switching between the two based on whether the error is formatted with%v
or%+v
.Allowing users to choose is just going to lead to more mess. One package uses
;
, another uses\n
, yet another,
. When I compose all of those errors together, the result is a big mess.A potential use-case for decomposing a multi-error is with non-fatal JSON (or protobuf) deserialization.
In such an application there could be a number of non-fatal errors where the user wants to have as much of the unmarshal process to proceed, but to remember every non-fatal error (e.g., mismatching types, invalid UTF-8, etc.). Afterwards, the caller inspects the set of errors to do additional work (e.g., extract the errors to present it more nicely in a UI, ignore some of them, do extra cleanup for some, etc.).
This doesn’t need to be solved using
Split
, but we definitely need a way to iterate through the errors in a multi-error. The JSON package could document that it is guaranteed to return a flattened list of errors such that a single call toerrors.Split
would return everything.A gotcha with multi-errors is that is that
errors.Is
is ill-defined as whether it meansexactly is
orcontains
. The former checks that every element in the multi-error is the target error. The latter checks whether any element in the multi-error is the target error. It’s a subtle difference, but has quite a significant impact.For example, there used to be a multi-error package in
google.golang.org/protobuf
to hold non-fatal errors (e.g., invalid UTF-8, unpopulated required fields, etc.).Suppose someone wanted to ignore required-not-set errors, it would be tempting do something like:
the intent of the user is to ignore only required-not-set errors, but to still reject other errors.
However, the seemingly innocent logic above is buggy if the error returned by
proto.Unmarshal
waserrors.Join(proto.ErrInvalidUTF8, proto.ErrRequiredNotSet)
as it would subtly ignore the presence of theproto.ErrInvalidUTF8
error.Regarding this issue, I don’t have any good solutions. There are times when I would want
errors.Is
to mean “exactly is” and other times that I want it to mean “contains”. Either semantic we choose, it’s going to be footgun.We avoided this problem by just not depending on multi-errors and to instead depend on options (e.g.,
proto.UnmarshalOptions.AllowPartial
) to prevent the generation ofproto.ErrRequiredNotSet
in the first place. This is one way to work around the issue, but I still think it would have been nice forproto.Unmarshal
to return a multi-error (e.g., a list of all errors annotated by the path and the error that occurred).Looking at that dog’s breakfast of outputs makes me think that the original Join proposal is the best of the lot. But I really don’t feel strongly.
I think this is the case for errors.Walk: most of the time you’re going to be logging into some sort of structured reporting system as opposed to just outputting text to stdout, so for that you’ll want the tree displayed as nested HTML tags or whatever. I think that if errors.Join just always used semicolons with %v and new lines plus bullets with %+v, that would get you a lot of the way towards formatting the stdout text correctly, and for any other more complicated case, it could be handled by Walk.
I think for that you want:
fmt.Errorf("THE HEADER: %w", errors.Join(errs))
.That’s how the Tailscale multierr package works. The reasoning is that it lets you write code that builds it incrementally:
In this case case, the final return will return nil, a single error, or a multierr composed purely of the non-nil errors encountered along the way.
I almost wrote this earlier. But I didn’t, when I realized that this is effectively equivalent to:
The only difference is that, as originally proposed, Join doesn’t return the sole non-nil error directly in the case in which there is exactly one non-nil error. But that could be changed. 😃
I agree with you very much, we should focus on the original core API supporting the multiple error model first, and then expose the complexity to the user. Before that, we can experiment with Join, Split, Walk, etc.
Most importantly, decide the definition of
Unwrap() []error
.I agree. Can I leave that for you to do? I’ve exceeded my time allotment for this for a while. 😃
We need to provide some reasonably convenient way to get at the underlying errors of a multierr. Perhaps
Split
isn’t the right API here (although I still feel pretty good about it), but we need something.Perhaps there’s a case for such an API, but I think it’s distinct from this proposal. Accessing the component errors of a multierr is different from a full tree walk.
Yes.
I find that the main reason I unwrap my own multierrors is so that I can report them separately to my logging service or display them as a list in a CLI. The original xerrors formatting proposal was dropped for being too complicated but I think that an easy to use error walking mechanism might make it easy to let users figure out how they want to display and format all the suberrors on their own.
Unspecified, but the contract for it says that the caller must not modify the slice, so it may.
Perhaps
Split
should copy the result into a new slice. This would leaveIs
andAs
potentially allocation-free, while adding a layer of safety (and an allocation) to the relatively unusual case of manually inspecting the individual errors.Is
andAs
are separate implementations for efficiency reasons, so there’s no internalEach
function. This proposal does make the case for an exportedEach
or similar stronger, since iteration becomes more complicated.We should probably wait and see what happens with generic iteration, however.