monstache: Lots of "Dropping invalid float64 value: NaN for document" but no fields are dropped
I’m using the prune-invalid-json option and receiving lots of warnings saying there is an invalid float64 in my document. I’ve seen that it could be a wrong date or something like that. I’ve double checked my MongoDB document but all dates and floats are valid. More strange than that is that, checking the document id the error prints to me it does not seems to have dropped any field!
Is there a way to know which field is triggering this error? I’ve also tried to parse one of those documents using JSON.parse through otto but it does everything flawlessly. My configuration can be seen below:
mongo-url = "<REDACTED>"
elasticsearch-urls = ["<REDACTED>"]
workers = ["T1"]
namespace-regex = "^offers_data.offers$"
mongo-oplog-collection-name = "oplog.$main"
resume = true
routing-namespaces = ["offers_data.offers"]
verbose = true
stats = true
pprof = false
prune-invalid-json = true
elasticsearch-max-bytes = 1024000000
elasticsearch-max-conns = 512
[gtm-settings]
channel-size = 4096
buffer-size = 512
buffer-duration = "150ms"
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 15 (7 by maintainers)
Well, I’ve found the problem, it was exactly what you said but related to another field that didn’t appeared before. The workaround I’ve found was the same I told you above:
Now I have no more annoying warnings in my monstache! Thank you very much for all the help and clarifications!