go: proposal: spec: add "must" operator # to check and return error

Instead of an operator, it could be a builtin generic function

If we’re having must as a builtin function, then it could also augment the error. i.e:

func fn() (..., error) {}

a := must(fn(), "extra error info")  // append/prepend string to error returned by fn

b := must(fn(), func(err error) error { /* return the "augmented" error */ })

IMO, this is much more readable than UserDeleteError(user)#DeleteUser(user).

Instead of an operator, it could be a builtin generic function with a signature like:

func must(T1, T2, ..., Tn, error) (T1, T2, ..., Tn)

So calling it just looks like:

result := must(foo(1,2,3))

(Edit: To be clear, I’m not expressing an opinion either way on the overall proposal. I’m just suggesting a less-intrusive syntax to the originally proposed # operator.)

My concern with “Must” would be that new developers would stop handling errors. They would just use the panic feature since it’s less typing. I already see tutorials trying to do this:

x, err := foo()
checkErr(err)

Where checkErr() is just a wrapper around panic:

func checkErr(e error) {
    if err != nil {
        panic(e)
    }
}

Making it easier to not correctly handle errors seems like it would be a big detriment to the language.

For what it’s worth, in the HotSpot JVM we had introduced (almost 20 years ago…) the C++ TRAPS and CHECK macros, trying to address a similar issue using the existing C++ language at that time (and trying to avoid C++ exception handling because compiler performance around functions with exception handling was unclear). The similarity with this proposal (though not notation) is striking. It’s interesting to see the same ideas come up again:

Every function that could possibly “raise an exception” (cause an error) ended with the parameter TRAPS in the signature; see e.g., http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/tip/src/share/vm/prims/jvm.cpp, line 116.

Every function call that invoked such a function, ended with a last argument CHECK in the function call (see the same file, line 127).

TRAPS essentially amounted to an extra parameter passed that was used to communicate an error condition. The CHECK macro translated, well, into the following glorious macro…

THREAD); if (HAS_PENDING_EXCEPTION) return       ; (0

(http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/9b0ca45cd756/src/share/vm/utilities/exceptions.hpp, line 183).

I leave it up to the reader to decipher the macros and judge their beauty…

With respect to this proposal, I think it’s actually more cryptic than the C++ approach above. Bike shedding aside, # doesn’t exactly shout readability. @minux already pointed out other drawbacks. Personally, I’m not a fan for Go (despite my past life involvement with the above).

How about introducing two keywords, fail and must ? fail as a block describes a common error handling pattern inside a function (kinda like a defer) that can be explicitly called (in which case the compiler just inlines the block in place) or implicitly called by the must() wrapper (when the called function returns a failure). must() looks like a function because it can have arguments (for error annotation mostly):

package main

import (
	"github.com/juju/errors"
)

func ParseEthernetHeader(data []byte) (mac *EthHeader, read int, err error) {
	fail (err error, reason string, args ...interface{}) {
		// first argument of a failure block is always the error
		return nil, 0, errors.Annotatef(err, reason, args...)
	}

	if len(data) < 8+6+6+3+4 {
		// one way to explicitly call the failure handler so we concentrate
		// error handling in a single place without relying on 'goto'
		fail(ErrPacketTooSmall, "packet with %d bytes too small to contain an ethernet frame", len(data))
	}
	mac = &EthHeader{}
	read += 8 // eth header preamble

	// parses a MAC address, if it fails, the fail() block is called automatically,
	// where `err` will be the error returned by ParseMAC and everything else is passed
	// as extra arguments (for annotations, etc)
	mac.Destination = must(ParseMAC(data[read:]), "invalid MAC address on destination field")
	read += 6 // source MAC
	// ...
	return mac, read, nil
}

func ParsePacket(ethernetFrame []byte) (*EthHeader, *IpHeader, *TcpHeader, *UdpHeader, []byte, error) {
	var ethernetHeader *EthHeader
	var ipHeader *IpHeader
	var tcpHeader *TcpHeader
	var udpHeader *UdpHeader
	var read int
	var totalRead int
	var payload []byte

	fail (err error) {
		// first argument of a failure block is always the error
		return nil, nil, nil, nil, []byte{}, err
	}

	ethernetHeader, read = must(ParseEthernetHeader(ethernetFrame[:]))
	totalRead += read
	ipHeader, read = must(ParseIPHeader(ethernetFrame[totalRead:]))
	totalRead += read

	switch ipHeader.Protocol {
	case 6:
		tcpHeader, read = must(ParseTcpHeader(ethernetFrame[totalRead:]))
		totalRead += read
	case 17:
		udpHeader, read = must(ParseUdpHeader(ethernetFrame[totalRead:]))
		totalRead += read
	default:
		fail(ErrProtocolUnsupported)
	}
	payload = ethernetFrame[totalRead:]
	payload = payload[:len(payload)-4]
	return ethernetHeader, ipHeader, tcpHeader, udpHeader, payload, nil
}

I like the idea, but share @Xeoncross concerns about making it easier to not handle errors properly

That aside, if we get it, I’d prefer must func() over must(func()) or the somewhat cryptic #func() to more closely match go and defer syntax

how about adding a keyword, roe (return on error), which like the go and defer keyword, being used as prefixes of function callings?

Hi,

I wonder if this change should be in the language at all or not. I know that many people think that the if err != nil { return err } is too much but I could argue that in fact it makes it clear to read quickly what the code does and it’s explicit about it.

We praise Go for the clarity and simplicity it brings to programming and we love it for being able to link articles that were written sometimes before 1.0 that are still relevant and apply to the language (for example: https://blog.golang.org/profiling-go-programs )

To this extent, I would propose this as change to the tooling surrounding Go rather than changing the language itself.

For example, for this example:

func test() (interface{}, error) {
	res, err := test()
	if err != nil {
		return nil, err
	}
	if res == nil {
		panic("Invalid resource")
	}
	return res, nil
}

this could look something like this: or could display the actual return / panic keywords instead of the symbols show in that picture.

This way, the users that want to make this “more” readable will be able to do so, while those who don’t will be able to keep the display settings as they are.

The other aspect of the proposal is already covered by most editors that support code templating as you could for example a template rerr that expand to if err != nil { return nil, err; } or perr that expands to if err != nil { panic($cursor$) } `.

One could also argue that the single line return / panic handling are not that common, depending on the software / libraries being used and that having people to actually think of error handling instead of bubbling this up is a feature not a problem. Or, as others shown, sometimes a comment is necessary to make it clear why a panic is needed instead of just returns.

As such I think this is much better suited for the tools to be change not the language and everyone gets what they need.

Not yet. I’d like to open a discussion about errors later this year.

The reason other proposals are rejected is not because of those fixable drawbacks.

The reason is fundamental: the proposed simplification encourages only those two kinds of error handling, but in fact, there is another: augmenting the error before returning to provide more contextual information. (You might say the proposal doesn’t preclude the other error handling strategy, but if it makes the first two so much easier to use, it will definitely reduce the incentive for people to even think about using the 3rd.)

Anyway, I think it’s possible to write a code transformer for this and see what happens.

I’m inclined to close this proposal but remember it for a longer error-related discussion later in the year (or just put it on hold until then). As I wrote at https://research.swtch.com/go2017#error, the big disappointment I have with the way most code uses errors is exactly that most code doesn’t add important context. It justs ‘return err’. Making that mistake even easier seems like a step in the wrong direction.

(And making it hard to see that whether code panics or returns an error is even worse, but that’s a relatively minor point.)

I was intrigued by @dlsniper’s suggestion, so I started a vim plugin to try it out: jargv/vim-go-error-folds. I’ve only used this very briefly, but so far I kind of like it.

Having written some examples in the documentation I tend to approach this proposal from the perspective of both using it in examples and impact on new gophers while reading them.

This would become, in my opinion, a very contentious operator/built-in, raising always a flamatory question on whether a such use case is more appropriate using it or not.

Frankly, I do not believe this should be a choice of the developer but of the package. If I, as a developer, must make such call, I can perfectly ending up with a divergent interpretation or decision from a fellow Gopher.

text/template and html/template are nice examples of what I am putting forward. If you happen to have a static template, that by definition shouldn’t fail - and it does fail, you can use template.Must to guard against this particular failure. You can preemptively protect from it by running a test using Must and have it again as a return simplifier in the real code.

It is no coincidence we don’t see Must functions spread around stdlib. I can imagine few miuses if applied to net or http.

In other words, I think this is a valid idea, but not for the language, only for specific packages that it might make sense.

We’re going to rethink errors in Go 2, so let’s leave the general topic for then. We’re very unlikely to add something this substantial before then.

@rsc at the very least, I think a callstack with file:line should always be part of error types… printing an error should show a full call stack (just like a panic).

Adding extra context to errors as a key-value would be ideal too (like context.Context) - so if inserting a row into a database table fails, I can add that row itself as part of the error (and my non-stdlib logging/metrics component saves that to elasticsearch, including a JSON representation of the column).

Most of the proposals above suggest a keyword in front of the function to be wrapped with error handling. I tend to disagree that this makes a nicer look or better readability than the if err := f(); err != nil... construct that is already available in Go.

I like how Perl’s ... or die idiom allows to put error handling at the end of the statement.

f() or die "f failed: $!"

(Where $! is like C’s errno.)

An equivalent in Go might be something like

f() or return nil, errors.Last("f failed")   
g() or log.Println(errors.Last("f failed"))

errors.Last() would take the error value returned by f() or g() and the string argument and return a new error object, just like errors.Wrap from the pkg/errors does.

Compared to:

err := f()
if err != nil {
    return nil, errors.New("f failed: " + err)
}
if err := g(); err != nil {
    log.Println(f failed:", err)
}

this approach provides some advantages:

• The code more concise without being cryptic.
• The code is still readable in a top-down fashion.
• The main control flow (i.e., the flow in the error-free case) is not disrupted by error handling statements.
• The main flow is not wrapped inside error handling; that is, error handling does not dominate the control flow.

Just to note: it’s not possible to add new keywords to Go 1. It will break programs where a new keyword is used as an identifier.

@bradfitz I would love to learn more about that rationale, because lack of ternary is another pain point for me. 😃 Makes me wonder if I’m missing some of the zeitgeist of Go (though I’m still a big fan!).

Lack of ternary is especially painful when I’m creating nested struct literals and I’ve got to put a conditional and a variable in order to conditionally populate some field, but now the declaration of the variable and its usage are potentially quit far away, whereas the ternary allows me to elide even creating a variable. Of course, I can change from declaratively creating the struct to imperatively creating it.

Actually come to think of it, declarative struct literals feel a bit like precedent for declarative ternary and declarative error handling. Is it idiomatic to avoid struct literals in go?

This is not a fully-fleshed out proposal, but here’s an observation on the minimal required language change: If the “must” operator 1. stripped off the err value if applicable and 2. returned the zero value of all non-err return values and passed the error through, you could get an awful lot of mileage out of:

func MayError(in Input) (out Output, err error) {
    defer func() {
        if err != nil {
            // in defer, full stack trace is still available
            err = WrapErrorWithContext(err)
        }
    }()

    out = must SomeOtherFunction()
    return
}

This pattern 1. allows wrapping return values and 2. allows must operator without a defer if the usual return mechanism is desired.

The core desire for “capturing” the error on the way out and doing something to it can be met with named parameters and defer already; in fact you can already use this technique now if you manually spell out the must with the corresponding if statement.

@Xeoncross said:

My concern with “Must” would be that new developers would stop handling errors. They would just use the panic feature since it’s less typing.

As an example of this exact issue, I saw this in Rust when I dabbled in it. In Rust, errors are returned as part of a tagged union called Result which can hold either the actual return value or an error. Result has a number of methods on it for handling errors, including an or() method, but most of the time people just call unwrap() which returns the return value if it’s not an error or panics if it is. Large amounts of the codebases I’ve looked through have unwrap()s everywhere and things panic all the time as a result. Of course, calling unwrap() everywhere is discouraged, but it’s so convenient that people do it all the time anyways planning to clean it up later.

As annoying and repetitive as Go error handling can get, I actually like that it forces you to handle errors when there could be one. It could certainly use some tweaking, such as to the difference between functions that just have an error return and functions that have multiple return values, but I think that whatever changes are made to it, it should always err on the annoying side rather than the convenient side.