go: testing: bad output formatting when test stdout ends in an incomplete line
What version of Go are you using (go version
)?
$ go version go version go1.14.1 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env
)?
go env
Output
$ go env GO111MODULE="on" GOARCH="amd64" GOBIN="" GOCACHE="/Users/pete/Library/Caches/go-build" GOENV="/Users/pete/Library/Application Support/go/env" GOEXE="" GOFLAGS=" -mod=" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GONOPROXY="github.com/circleci" GONOSUMDB="github.com/circleci" GOOS="darwin" GOPATH="/Users/pete/Source/go" GOPRIVATE="github.com/circleci" GOPROXY="direct" GOROOT="/usr/local/Cellar/go/1.14.1/libexec" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/Cellar/go/1.14.1/libexec/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="<<SOMETHING>>/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lh/8qc651cn25n1knh824hcyd0r0000gn/T/go-build757047380=/tmp/go-build -gno-record-gcc-switches -fno-common"
What did you do?
package testissue
import (
"fmt"
"testing"
)
func TestBad(t *testing.T) {
// No newline below
fmt.Print("bad output")
}
func TestGood(t *testing.T) {
// Print a newline below
fmt.Println("good output")
}
then run:
go test -json ./testissue
What is the issue?
The JSON output seems corrupted for the “bad” test. You can see that it is missing a “pass” action.
The issue appears to be in the conversion the test2json
command is doing from stdout of the test runner into JSON. https://github.com/golang/go/blob/master/src/cmd/internal/test2json/test2json.go
{"Time":"2020-03-25T10:07:16.831092Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestBad"}
{"Time":"2020-03-25T10:07:16.83137Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"=== RUN TestBad\n"}
{"Time":"2020-03-25T10:07:16.831384Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestBad","Output":"bad output--- PASS: TestBad (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831401Z","Action":"run","Package":"github.com/circleci/distributor/testissue","Test":"TestGood"}
{"Time":"2020-03-25T10:07:16.831405Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"=== RUN TestGood\n"}
{"Time":"2020-03-25T10:07:16.831414Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"good output\n"}
{"Time":"2020-03-25T10:07:16.831423Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Output":"--- PASS: TestGood (0.00s)\n"}
{"Time":"2020-03-25T10:07:16.831426Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Test":"TestGood","Elapsed":0}
{"Time":"2020-03-25T10:07:16.831443Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"PASS\n"}
{"Time":"2020-03-25T10:07:16.831486Z","Action":"output","Package":"github.com/circleci/distributor/testissue","Output":"ok \tgithub.com/circleci/distributor/testissue\t0.149s\n"}
{"Time":"2020-03-25T10:07:16.833206Z","Action":"pass","Package":"github.com/circleci/distributor/testissue","Elapsed":0.151}
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 16 (12 by maintainers)
Commits related to this issue
- test: add extra newline to error message This is a workaround for https://github.com/golang/go/issues/38063, where Go incorrectly reports test results if the test output doesn't end in newlines — committed to tilt-dev/tilt by nicks 3 years ago
- test: add extra newline to error message (#4232) This is a workaround for https://github.com/golang/go/issues/38063, where Go incorrectly reports test results if the test output doesn't end in newl... — committed to tilt-dev/tilt by nicks 3 years ago
Maybe. Test binaries write messages to the
go test
driver onstdout
. Tests may also write tostdout
, so the output gets mixed together. Test binaries could write messages on some other channel (pipe, socket, whatever) to avoid this problem.@nicks The issue remains unsolved, but not really sure if someone has the interest to write a fix that well balances the implementation complexity and the actual achievements on solving the problem (and probably the reason that @pete-woods of this issue closes this). As for now, a simple workaround is as you referenced: add a
\n
if needed.@changkun,
testing
andos
are both in the standard library, so if need be we could even hookos.Stdout
andos.Stderr
with a side-channel to tell thetesting
package about output line state.(#29062 is closely related, in the sense that it could be implemented by hooking the
os
package.)Unfortunately not. Each package’s test is built into a separate binary and run in a separate process. Global side effects from tests and
init
functions shouldn’t be visible to tests for other packages.