fabio: runtime error: integer divide by zero
Hi,
I’ve run into a bug. When running a service consisting of around 100 containers this error does not occur, but scaling up to several 100’s it does. Other services with different urlprefix still continue to work.
Nov 15 09:47:58 random-host fabio[471]: route add local-bla-app hello-world.domain.com/ http://192.168.10.55:8080/ tags "random-host-urlprefix-hello-world.domain.com/"
Nov 15 09:48:37 random-host fabio[471]: 2016/11/15 09:48:37 http: panic serving [::1]:46124: runtime error: integer divide by zero
Nov 15 09:48:37 random-host fabio[471]: goroutine 28 [running]:
Nov 15 09:48:37 random-host fabio[471]: net/http.(*conn).serve.func1(0xc421e0c400)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/go1.7.3/src/net/http/server.go:1491 +0x12a
Nov 15 09:48:37 random-host fabio[471]: panic(0x7d13c0, 0xc42000e070)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/go1.7.3/src/runtime/panic.go:458 +0x243
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/route.glob..func1(0x0, 0xc421d44101)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/route/picker.go:48 +0x92
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/route.rndPicker(0xc42004f680, 0x1)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/route/picker.go:30 +0x36
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/route.Table.lookup(0xc421d420c0, 0xc421e102b0, 0xf, 0xc421e10294, 0x1, 0x0, 0x0, 0x6563617254)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/route/table.go:281 +0x3ea
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/route.Table.Lookup(0xc421d420c0, 0xc4200b33b0, 0x0, 0x0, 0x0)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/route/table.go:253 +0x226
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/proxy.target(0xc4200b33b0, 0x7f5f2453b401)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/proxy/util.go:94 +0x8b
Nov 15 09:48:37 random-host fabio[471]: github.com/eBay/fabio/proxy.(*httpProxy).ServeHTTP(0xc42019e0f0, 0xa1d900, 0xc420165040, 0xc4200b33b0)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/gopath/src/github.com/eBay/fabio/proxy/proxy.go:35 +0x9c
Nov 15 09:48:37 random-host fabio[471]: net/http.serverHandler.ServeHTTP(0xc420070100, 0xa1d900, 0xc420165040, 0xc4200b33b0)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/go1.7.3/src/net/http/server.go:2202 +0x7d
Nov 15 09:48:37 random-host fabio[471]: net/http.(*conn).serve(0xc421e0c400, 0xa1e100, 0xc420c847c0)
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/go1.7.3/src/net/http/server.go:1579 +0x4b7
Nov 15 09:48:37 random-host fabio[471]: created by net/http.(*Server).Serve
Nov 15 09:48:37 random-host fabio[471]: /Users/frschroeder/go1.7.3/src/net/http/server.go:2293 +0x44d
Nov 15 09:48:57 random-host fabio[471]: 2016/11/15 09:48:57 [INFO] consul: Health changed to #107221
Nov 15 09:49:35 random-host fabio[471]: 2016/11/15 09:49:35 [INFO] consul: Health changed to #107231
Nov 15 09:49:42 random-host fabio[471]: 2016/11/15 09:49:42 [INFO] consul: Health changed to #107233
Nov 15 09:49:44 random-host fabio[471]: 2016/11/15 09:49:44 [INFO] consul: Health changed to #107235
Nov 15 09:49:44 random-host fabio[471]: 2016/11/15 09:49:44 [INFO] consul: Health changed to #107396
Nov 15 09:49:44 random-host fabio[471]: 2016/11/15 09:49:44 [INFO] Updated config to
Nov 15 09:49:44 random-host fabio[471]: route add local-bla-app hello-world.domain.com/ http://192.168.8.8:8080/ tags "random-host-urlprefix-hello-world.domain.com/"
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 15 (10 by maintainers)
Commits related to this issue
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
- Issue #186: runtime error: integer divide by zero When fabio is presented with more than 200 targets for a single route then a rounding error had the effect that no target would get traffic which sub... — committed to fabiolb/fabio by magiconair 8 years ago
You’ve indeed found a bug and it is in code for which I don’t even have a test… How embarrassing.
What happens is that the weighing algorithm which pre-computes the distribution of the target URLs based on weight allocates only 100 slots (= 100%) and when you create more than a couple hundred instances the rounding errors ensure that no instance is used. This then trips the random number function which never thought it would be called with zero. Why would anybody create more than 100 instances anyway? 😉
Time to refactor. Not a biggie but I need to add a test first. For now stay <= 100 instances per prefix.
I’ve got an ugly patch which you can apply if this is an urgent production issue. Otherwise, a fix is coming soon.
Thanks for finding this.