go: testing: bad output formatting when test stdout ends in an incomplete line
What version of Go are you using (go version)?
$ go version go version go1.14.1 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env GO111MODULE="on" GOARCH="amd64" GOBIN="" GOCACHE="/Users/pete/Library/Caches/go-build" GOENV="/Users/pete/Library/Application Support/go/env" GOEXE="" GOFLAGS=" -mod=" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GONOPROXY="github.com/circleci" GONOSUMDB="github.com/circleci" GOOS="darwin" GOPATH="/Users/pete/Source/go" GOPRIVATE="github.com/circleci" GOPROXY="direct" GOROOT="/usr/local/Cellar/go/1.14.1/libexec" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/Cellar/go/1.14.1/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="<<SOMETHING>>/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lh/8qc651cn25n1knh824hcyd0r0000gn/T/go-build757047380=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
package testissue
import (
"fmt"
"testing"
)
func TestBad(t *testing.T) {
// No newline below
fmt.Print("bad output")
}
func TestGood(t *testing.T) {
// Print a newline below
fmt.Println("good output")
}
then run:
go test -json ./testissue
What is the issue?
The JSON output seems corrupted for the “bad” test. You can see that it is missing a “pass” action.
The issue appears to be in the conversion the test2json command is doing from stdout of the test runner into JSON. https://github.com/golang/go/blob/master/src/cmd/internal/test2json/test2json.go
{"Time":"2020-03-25T10:07:16.831092Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestBad"}
{"Time":"2020-03-25T10:07:16.83137Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"=== RUN TestBad\n"}
{"Time":"2020-03-25T10:07:16.831384Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"bad output--- PASS: TestBad (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831401Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestGood"}
{"Time":"2020-03-25T10:07:16.831405Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"=== RUN TestGood\n"}
{"Time":"2020-03-25T10:07:16.831414Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"good output\n"}
{"Time":"2020-03-25T10:07:16.831423Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"--- PASS: TestGood (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831426Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Elapsed":0}
{"Time":"2020-03-25T10:07:16.831443Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"PASS\n"}
{"Time":"2020-03-25T10:07:16.831486Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"ok \tgithub.com/circleci/distributor/testissue\t0.149s\n"}
{"Time":"2020-03-25T10:07:16.833206Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Elapsed":0.151}
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 16 (12 by maintainers)
Commits related to this issue
- test: add extra newline to error message This is a workaround for https://github.com/golang/go/issues/38063, where Go incorrectly reports test results if the test output doesn't end in newlines — committed to tilt-dev/tilt by nicks 3 years ago
- test: add extra newline to error message (#4232) This is a workaround for https://github.com/golang/go/issues/38063, where Go incorrectly reports test results if the test output doesn't end in newl... — committed to tilt-dev/tilt by nicks 3 years ago
Maybe. Test binaries write messages to the
go testdriver onstdout. Tests may also write tostdout, so the output gets mixed together. Test binaries could write messages on some other channel (pipe, socket, whatever) to avoid this problem.@nicks The issue remains unsolved, but not really sure if someone has the interest to write a fix that well balances the implementation complexity and the actual achievements on solving the problem (and probably the reason that @pete-woods of this issue closes this). As for now, a simple workaround is as you referenced: add a
\nif needed.@changkun,
testingandosare both in the standard library, so if need be we could even hookos.Stdoutandos.Stderrwith a side-channel to tell thetestingpackage about output line state.(#29062 is closely related, in the sense that it could be implemented by hooking the
ospackage.)Unfortunately not. Each package’s test is built into a separate binary and run in a separate process. Global side effects from tests and
initfunctions shouldn’t be visible to tests for other packages.