monstache: Lots of "Dropping invalid float64 value: NaN for document" but no fields are dropped

I’m using the prune-invalid-json option and receiving lots of warnings saying there is an invalid float64 in my document. I’ve seen that it could be a wrong date or something like that. I’ve double checked my MongoDB document but all dates and floats are valid. More strange than that is that, checking the document id the error prints to me it does not seems to have dropped any field!

Is there a way to know which field is triggering this error? I’ve also tried to parse one of those documents using JSON.parse through otto but it does everything flawlessly. My configuration can be seen below:

    mongo-url = "<REDACTED>"
    elasticsearch-urls = ["<REDACTED>"]
    workers = ["T1"]
    namespace-regex = "^offers_data.offers$"
    mongo-oplog-collection-name = "oplog.$main"
    resume = true
    routing-namespaces = ["offers_data.offers"]

    verbose = true
    stats = true
    pprof = false
    prune-invalid-json = true

    elasticsearch-max-bytes = 1024000000
    elasticsearch-max-conns = 512

    [gtm-settings]
    channel-size = 4096
    buffer-size = 512
    buffer-duration = "150ms"

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

Well, I’ve found the problem, it was exactly what you said but related to another field that didn’t appeared before. The workaround I’ve found was the same I told you above:

var date1 = new Date(JSON.stringify(doc.departure_date).replace(/\"/g,""))

Now I have no more annoying warnings in my monstache! Thank you very much for all the help and clarifications!