go: strconv: FormatFloat rounding error

What version of Go are you using (go version)?


go version go1.11.4 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Anders\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Anders\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Anders\AppData\Local\Temp\go-build2129905
62=/tmp/go-build -gno-record-gcc-switches

What did you do?

I’m trying to implement a JSON canonicalization scheme requiring numbers to be expressed in the shortest way. prec -1 seemed like a better alternative than porting V8’s solution but unfortunately it seems that the -1 option is broken:

package main

import (
    "strconv"
    "math"
    "fmt"
)

func fpformat(hex string, format byte, precision int, expected string) {
    ieeeU64, err := strconv.ParseUint(hex, 16, 64)
    if err != nil {
        panic(hex)
    }
    ieeeF64 := math.Float64frombits(ieeeU64)
    shortestNumber := strconv.FormatFloat(ieeeF64, format, -1, 64)
    moreDecimals := strconv.FormatFloat(ieeeF64, format, precision, 64)
    plusOneDecimal := strconv.FormatFloat(ieeeF64, format, precision + 1, 64)
    fmt.Printf("\nfloat64hex: %s\n", hex)
    fmt.Printf("   actual using precision[-1]: %s\n", shortestNumber)  // INCORRECT in go1.11.4 windows/amd64
    fmt.Printf(" expected using precision[-1]: %s\n", expected)
    fmt.Printf("   actual using precision[%2d]: %s\n", precision, moreDecimals)
    fmt.Printf("   actual using precision[%2d]: %s\n", precision + 1, plusOneDecimal)
}

func main() {
    fpformat("439babe4b56e8a39", 'f', 0, "498484681984085570")
    fpformat("c4dee27bdef22651", 'g', 17, "-5.8339553793802237e+23")
}

What did you expect to see?

float64hex: 439babe4b56e8a39
   actual using precision[-1]: 498484681984085570
 expected using precision[-1]: 498484681984085570
   actual using precision[ 0]: 498484681984085568
   actual using precision[ 1]: 498484681984085568.0

float64hex: c4dee27bdef22651
   actual using precision[-1]: -5.8339553793802237e+23
 expected using precision[-1]: -5.8339553793802237e+23
   actual using precision[17]: -5.8339553793802237e+23
   actual using precision[18]: -5.83395537938022366e+23

What did you see instead?

float64hex: 439babe4b56e8a39
   actual using precision[-1]: 498484681984085560
 expected using precision[-1]: 498484681984085570
   actual using precision[ 0]: 498484681984085568
   actual using precision[ 1]: 498484681984085568.0

float64hex: c4dee27bdef22651
   actual using precision[-1]: -5.8339553793802236e+23
 expected using precision[-1]: -5.8339553793802237e+23
   actual using precision[17]: -5.8339553793802237e+23
   actual using precision[18]: -5.83395537938022366e+23

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 17 (7 by maintainers)

Commits related to this issue

Most upvoted comments

Apparently JavaScript agree:

Number.parseFloat("255.255").toPrecision(5)
> "255.25"

The rounding is performed at the binary level which is one reason to why floating point is not suitable for monetary amounts

@cyberphone that’s just a bug in the Java code. See https://github.com/ulfjack/ryu/issues/83. My Ryu code gives 5e-324.