splunk-connect-for-kubernetes: Missing Logs

I have the below settings in my configmap, I am seeing loss of data in splunk ,not all the logs are pushed to splunk,

buffer:
  "@type": memory
  total_limit_size: 500m
  chunk_limit_size: 100m
  chunk_limit_records: 20000
  flush_interval: 5s
  flush_thread_count: 8
  overflow_action: drop_oldest_chunk
  #retry_max_times: 10
  retry_forever: true
  retry_type: periodic
  retry_wait: 5s

Could someone guide me if I am missing something here, I have observed loss of logs on only specific nodes. My setup is a on prem k8s with 3 masters and 10 nodes,kubernetes version is 15.5