smokeping_prober: panic: Value kind is Duration, not Time
We’ve got a smokeping_prober docker container that is dying with this error message. Here’s one recent example, with hostnames and IP addresses changed to protect the innocent:
2025-04-03T15:50:34.021208361Z time=2025-04-03T15:50:34.021Z level=INFO source=main.go:228 msg="Starting smokeping_prober" version="(version=0.9.0, branch=HEAD, revision=1c1bd4704bfa7254e898025907a7582c6df284e1)"
2025-04-03T15:50:34.021267761Z time=2025-04-03T15:50:34.021Z level=INFO source=main.go:229 msg="Build context" build_context="(go=go1.23.5, platform=linux/amd64, user=root@c3ae5db9db58, date=20250121-16:38:33, tags=unknown)"
2025-04-03T15:50:34.021596945Z time=2025-04-03T15:50:34.021Z level=INFO source=main.go:237 msg="Leaving GOMAXPROCS=8: CPU quota undefined" component=automaxprocs
2025-04-03T15:50:34.029490512Z time=2025-04-03T15:50:34.029Z level=INFO source=main.go:109 msg="Starting prober" address=localhost interval=1s size_bytes=56 source_address=127.0.0.1
2025-04-03T15:50:34.079939153Z time=2025-04-03T15:50:34.079Z level=INFO source=main.go:109 msg="Starting prober" address=192.168.10.20 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.130627987Z time=2025-04-03T15:50:34.130Z level=INFO source=main.go:109 msg="Starting prober" address=192.168.10.30 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.180875477Z time=2025-04-03T15:50:34.180Z level=INFO source=main.go:109 msg="Starting prober" address=host1.example.com interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.231518661Z time=2025-04-03T15:50:34.231Z level=INFO source=main.go:109 msg="Starting prober" address=172.16.10.20 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.281877042Z time=2025-04-03T15:50:34.281Z level=INFO source=main.go:109 msg="Starting prober" address=172.16.10.30 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.332441354Z time=2025-04-03T15:50:34.332Z level=INFO source=main.go:109 msg="Starting prober" address=host2.example.com interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.382761425Z time=2025-04-03T15:50:34.382Z level=INFO source=main.go:109 msg="Starting prober" address=192.168.100.200 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.432803963Z time=2025-04-03T15:50:34.432Z level=INFO source=main.go:109 msg="Starting prober" address=192.168.100.300 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.483868690Z time=2025-04-03T15:50:34.483Z level=INFO source=main.go:109 msg="Starting prober" address=host3.example.com interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.534916428Z time=2025-04-03T15:50:34.534Z level=INFO source=main.go:109 msg="Starting prober" address=172.16.20.10 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.585506941Z time=2025-04-03T15:50:34.585Z level=INFO source=main.go:109 msg="Starting prober" address=172.16.30.40 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.635850101Z time=2025-04-03T15:50:34.635Z level=INFO source=main.go:109 msg="Starting prober" address=172.16.40.10 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.686441674Z time=2025-04-03T15:50:34.686Z level=INFO source=main.go:109 msg="Starting prober" address=host4.example.com interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.737325021Z time=2025-04-03T15:50:34.737Z level=INFO source=main.go:109 msg="Starting prober" address=10.10.20.30 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.787258637Z time=2025-04-03T15:50:34.787Z level=INFO source=main.go:109 msg="Starting prober" address=host1.example.net interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.838254225Z time=2025-04-03T15:50:34.838Z level=INFO source=main.go:109 msg="Starting prober" address=10.10.20.40 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.888978009Z time=2025-04-03T15:50:34.888Z level=INFO source=main.go:109 msg="Starting prober" address=host2.example.net interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.939711914Z time=2025-04-03T15:50:34.939Z level=INFO source=main.go:109 msg="Starting prober" address=10.10.20.50 interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:34.990631470Z time=2025-04-03T15:50:34.990Z level=INFO source=main.go:109 msg="Starting prober" address=host3.example.net interval=1s size_bytes=56 source_address=""
2025-04-03T15:50:35.042488595Z time=2025-04-03T15:50:35.042Z level=INFO source=tls_config.go:347 msg="Listening on" address=[::]:9374
2025-04-03T15:50:35.042514496Z time=2025-04-03T15:50:35.042Z level=INFO source=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9374
2025-04-03T15:50:35.083051411Z panic: Value kind is Duration, not Time
2025-04-03T15:50:35.083095901Z
2025-04-03T15:50:35.083107531Z goroutine 55 [running]:
2025-04-03T15:50:35.083119391Z log/slog.Value.Time({{}, 0xc0002be3c0?, {0x9e8c00?, 0xb96fb8?}})
2025-04-03T15:50:35.083128282Z /usr/local/go/src/log/slog/value.go:379 +0x168
2025-04-03T15:50:35.083137802Z github.com/prometheus/common/promslog.init.func2({0xc0000f7608?, 0x9e8c00?, 0xb96fb8?}, {{0xaad504, 0x4}, {{}, 0x2a76b, {0x9e8c00, 0xb96fb8}}})
2025-04-03T15:50:35.083145952Z /go/pkg/mod/github.com/prometheus/common@v0.62.0/promslog/slog.go:75 +0x90
2025-04-03T15:50:35.083153832Z log/slog.(*handleState).appendAttr(0xc0000f7608, {{0xaad504, 0x4}, {{}, 0x2a76b, {0x9e8c00, 0xb96fb8}}})
2025-04-03T15:50:35.083162512Z /usr/local/go/src/log/slog/handler.go:475 +0x21e
2025-04-03T15:50:35.083170562Z log/slog.(*handleState).appendNonBuiltIns.func1({{0xaad504, 0x4}, {{}, 0x2a76b, {0x9e8c00, 0xb96fb8}}})
2025-04-03T15:50:35.083204082Z /usr/local/go/src/log/slog/handler.go:342 +0x52
2025-04-03T15:50:35.083237073Z log/slog.Record.Attrs({{0xc1f3c972c4d096a6, 0x3f4c2681, 0xfcbf40}, {0xab26e5, 0xa}, 0xfffffffffffffffc, 0x97eba7, {{{0xaafa68, 0x7}, {{...}, ...}}, ...}, ...}, ...)
2025-04-03T15:50:35.083266373Z /usr/local/go/src/log/slog/record.go:84 +0xaa
2025-04-03T15:50:35.083313023Z log/slog.(*handleState).appendNonBuiltIns(_, {{0xc1f3c972c4d096a6, 0x3f4c2681, 0xfcbf40}, {0xab26e5, 0xa}, 0xfffffffffffffffc, 0x97eba7, {{{0xaafa68, 0x7}, ...}, ...}, ...})
2025-04-03T15:50:35.083337844Z /usr/local/go/src/log/slog/handler.go:341 +0x4c7
2025-04-03T15:50:35.083372584Z log/slog.(*commonHandler).handle(_, {{0xc1f3c972c4d096a6, 0x3f4c2681, 0xfcbf40}, {0xab26e5, 0xa}, 0xfffffffffffffffc, 0x97eba7, {{{0xaafa68, 0x7}, ...}, ...}, ...})
2025-04-03T15:50:35.083392854Z /usr/local/go/src/log/slog/handler.go:308 +0x5a5
2025-04-03T15:50:35.083424304Z log/slog.(*TextHandler).Handle(_, {_, _}, {{0xc1f3c972c4d096a6, 0x3f4c2681, 0xfcbf40}, {0xab26e5, 0xa}, 0xfffffffffffffffc, 0x97eba7, ...})
2025-04-03T15:50:35.083437644Z /usr/local/go/src/log/slog/text_handler.go:95 +0x3f
2025-04-03T15:50:35.083469355Z log/slog.(*Logger).log(0xc000028600, {0xba1d90, 0xfee3a0}, 0xfffffffffffffffc, {0xab26e5, 0xa}, {0xc0001d2c20, 0xa, 0xa})
2025-04-03T15:50:35.083486045Z /usr/local/go/src/log/slog/logger.go:257 +0x1e4
2025-04-03T15:50:35.083494345Z log/slog.(*Logger).Debug(...)
2025-04-03T15:50:35.083511505Z /usr/local/go/src/log/slog/logger.go:200
2025-04-03T15:50:35.083523865Z main.(*SmokepingCollector).updatePingers.func1(0xc000034d80)
2025-04-03T15:50:35.083534755Z /app/collector.go:121 +0x3ad
2025-04-03T15:50:35.083545105Z github.com/prometheus-community/pro-bing.(*Pinger).processPacket(0xc0002563c0, 0xc000034d40)
2025-04-03T15:50:35.083555686Z /go/pkg/mod/github.com/prometheus-community/pro-bing@v0.6.0/ping.go:870 +0x58f
2025-04-03T15:50:35.083565706Z github.com/prometheus-community/pro-bing.(*Pinger).runLoop(0xc0002563c0, {0xba7a80, 0xc0001866a8}, 0xc0001beee0)
2025-04-03T15:50:35.083576386Z /go/pkg/mod/github.com/prometheus-community/pro-bing@v0.6.0/ping.go:630 +0x2f3
2025-04-03T15:50:35.083586596Z github.com/prometheus-community/pro-bing.(*Pinger).run.func3()
2025-04-03T15:50:35.083596636Z /go/pkg/mod/github.com/prometheus-community/pro-bing@v0.6.0/ping.go:595 +0x59
2025-04-03T15:50:35.083606846Z golang.org/x/sync/errgroup.(*Group).Go.func1()
2025-04-03T15:50:35.083618406Z /go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78 +0x50
2025-04-03T15:50:35.083629116Z created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 52
2025-04-03T15:50:35.083639156Z /go/pkg/mod/golang.org/x/sync@v0.10.0/errgroup/errgroup.go:75 +0x96
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (3 by maintainers)
Commits related to this issue
- Fix: Rename time logging field Avoid conflict with the existing log line `time` field by renamign to `rtt`. * Also add missing TOS to debug logging. Fixes: https://github.com/SuperQ/smokeping_prober... — committed to SuperQ/smokeping_prober by SuperQ 2 years ago
- Fix: Rename time logging field Avoid conflict with the existing log line `time` field by renaming to `rtt`. * Also add missing TOS to debug logging. Fixes: https://github.com/SuperQ/smokeping_prober... — committed to SuperQ/smokeping_prober by SuperQ 2 years ago
latestWill be updated when I cut a new release. You can pin to a specific master tag likequay.io/superq/smokeping-prober@sha256:8d55886e44e2051f018f756e1642ae312aa8c21f2da801636507b625d283988fas a temporary workaround.Yea,
timeis the problem because it’s a conflicting key with thetimein the log line. This was fixed in the upstream Prometheus logging wrapper. With the latestprometheus/commmon, the logs look like this:@bknowles deferring to @SuperQ here, but it seems like some combination of issues from the adoption of slog is the issue, or perhaps, the inheritance of slog from the prometheus common packages?
fwiw, as far as I know, slog expects “time” fields in a log to be of type
Time.timehttps://pkg.go.dev/golang.org/x/exp/slog#pkg-constants
However, here it’s being called with
pkt.Rtthttps://github.com/SuperQ/smokeping_prober/blob/4dd0082212607983f84885d7e64803a75650f8b1/collector.go#L123-L125
I believe
pkt.Rttin this case istime.Durationhttps://github.com/prometheus-community/pro-bing/blob/c9b2c133ccf6c6212f29ca33c6258f7400ea76f6/ping.go#L257-L259Hope that helps. appreciate all the work y’all do here @SuperQ to keep things running smoothly and w/prom in general!