go: strconv: FormatFloat rounding error
What version of Go are you using (go version
)?
go version go1.11.4 windows/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env
)?
go env
Output
$ set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\Anders\AppData\Local\go-build set GOEXE=.exe set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOOS=windows set GOPATH=C:\Users\Anders\go set GOPROXY= set GORACE= set GOROOT=C:\Go set GOTMPDIR= set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64 set GCCGO=gccgo set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD= set CGO_CFLAGS=-g -O2 set CGO_CPPFLAGS= set CGO_CXXFLAGS=-g -O2 set CGO_FFLAGS=-g -O2 set CGO_LDFLAGS=-g -O2 set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Anders\AppData\Local\Temp\go-build2129905 62=/tmp/go-build -gno-record-gcc-switches
What did you do?
I’m trying to implement a JSON canonicalization scheme requiring numbers to be expressed in the shortest way. prec -1 seemed like a better alternative than porting V8’s solution but unfortunately it seems that the -1 option is broken:
package main
import (
"strconv"
"math"
"fmt"
)
func fpformat(hex string, format byte, precision int, expected string) {
ieeeU64, err := strconv.ParseUint(hex, 16, 64)
if err != nil {
panic(hex)
}
ieeeF64 := math.Float64frombits(ieeeU64)
shortestNumber := strconv.FormatFloat(ieeeF64, format, -1, 64)
moreDecimals := strconv.FormatFloat(ieeeF64, format, precision, 64)
plusOneDecimal := strconv.FormatFloat(ieeeF64, format, precision + 1, 64)
fmt.Printf("\nfloat64hex: %s\n", hex)
fmt.Printf(" actual using precision[-1]: %s\n", shortestNumber) // INCORRECT in go1.11.4 windows/amd64
fmt.Printf(" expected using precision[-1]: %s\n", expected)
fmt.Printf(" actual using precision[%2d]: %s\n", precision, moreDecimals)
fmt.Printf(" actual using precision[%2d]: %s\n", precision + 1, plusOneDecimal)
}
func main() {
fpformat("439babe4b56e8a39", 'f', 0, "498484681984085570")
fpformat("c4dee27bdef22651", 'g', 17, "-5.8339553793802237e+23")
}
What did you expect to see?
float64hex: 439babe4b56e8a39 actual using precision[-1]: 498484681984085570 expected using precision[-1]: 498484681984085570 actual using precision[ 0]: 498484681984085568 actual using precision[ 1]: 498484681984085568.0 float64hex: c4dee27bdef22651 actual using precision[-1]: -5.8339553793802237e+23 expected using precision[-1]: -5.8339553793802237e+23 actual using precision[17]: -5.8339553793802237e+23 actual using precision[18]: -5.83395537938022366e+23
What did you see instead?
float64hex: 439babe4b56e8a39 actual using precision[-1]: 498484681984085560 expected using precision[-1]: 498484681984085570 actual using precision[ 0]: 498484681984085568 actual using precision[ 1]: 498484681984085568.0 float64hex: c4dee27bdef22651 actual using precision[-1]: -5.8339553793802236e+23 expected using precision[-1]: -5.8339553793802237e+23 actual using precision[17]: -5.8339553793802237e+23 actual using precision[18]: -5.83395537938022366e+23
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 17 (7 by maintainers)
Commits related to this issue
- strconv: fix rounding in FormatFloat fallback path Float formatting uses a multiprecision fallback path where Grisu3 algorithm fails. This has a bug during the rounding phase: the difference between ... — committed to remyoudompheng/go by cespare 5 years ago
- strconv: fix rounding in FormatFloat fallback path Float formatting uses a multiprecision fallback path where Grisu3 algorithm fails. This has a bug during the rounding phase: the difference between ... — committed to remyoudompheng/go by cespare 5 years ago
Apparently JavaScript agree:
The rounding is performed at the binary level which is one reason to why floating point is not suitable for monetary amounts
@cyberphone that’s just a bug in the Java code. See https://github.com/ulfjack/ryu/issues/83. My Ryu code gives 5e-324.