go: testing: bad output formatting when test stdout ends in an incomplete line

What version of Go are you using (go version)?

$ go version
go version go1.14.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/pete/Library/Caches/go-build"
GOENV="/Users/pete/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=" -mod="
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="github.com/circleci"
GONOSUMDB="github.com/circleci"
GOOS="darwin"
GOPATH="/Users/pete/Source/go"
GOPRIVATE="github.com/circleci"
GOPROXY="direct"
GOROOT="/usr/local/Cellar/go/1.14.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="<<SOMETHING>>/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lh/8qc651cn25n1knh824hcyd0r0000gn/T/go-build757047380=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

package testissue

import (
	"fmt"
	"testing"
)

func TestBad(t *testing.T) {
	// No newline below
	fmt.Print("bad output")
}

func TestGood(t *testing.T) {
	// Print a newline below
	fmt.Println("good output")
}

then run:

go test -json ./testissue

What is the issue?

The JSON output seems corrupted for the “bad” test. You can see that it is missing a “pass” action.

The issue appears to be in the conversion the test2json command is doing from stdout of the test runner into JSON. https://github.com/golang/go/blob/master/src/cmd/internal/test2json/test2json.go

{"Time":"2020-03-25T10:07:16.831092Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestBad"}
{"Time":"2020-03-25T10:07:16.83137Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"=== RUN   TestBad\n"}
{"Time":"2020-03-25T10:07:16.831384Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"bad output--- PASS: TestBad (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831401Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestGood"}
{"Time":"2020-03-25T10:07:16.831405Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"=== RUN   TestGood\n"}
{"Time":"2020-03-25T10:07:16.831414Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"good output\n"}
{"Time":"2020-03-25T10:07:16.831423Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"--- PASS: TestGood (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831426Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Elapsed":0}
{"Time":"2020-03-25T10:07:16.831443Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"PASS\n"}
{"Time":"2020-03-25T10:07:16.831486Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"ok  \tgithub.com/circleci/distributor/testissue\t0.149s\n"}
{"Time":"2020-03-25T10:07:16.833206Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Elapsed":0.151}

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (12 by maintainers)

Commits related to this issue

Most upvoted comments

Can we really do anything about this? I could not imagine a way to determine the last character from flushed outputs…

Maybe. Test binaries write messages to the go test driver on stdout. Tests may also write to stdout, so the output gets mixed together. Test binaries could write messages on some other channel (pipe, socket, whatever) to avoid this problem.

@nicks The issue remains unsolved, but not really sure if someone has the interest to write a fix that well balances the implementation complexity and the actual achievements on solving the problem (and probably the reason that @pete-woods of this issue closes this). As for now, a simple workaround is as you referenced: add a \n if needed.

@changkun, testing and os are both in the standard library, so if need be we could even hook os.Stdout and os.Stderr with a side-channel to tell the testing package about output line state.

(#29062 is closely related, in the sense that it could be implemented by hooking the os package.)

Could it even be kept in-process, to avoid the serialisation overhead?

Unfortunately not. Each package’s test is built into a separate binary and run in a separate process. Global side effects from tests and init functions shouldn’t be visible to tests for other packages.