VictoriaMetrics: High memory footprint for vmagent v1.55.1

After upgrading from v1.53.1 to v1.55.1, I see a high memory footprint of vmagent.

Screenshots If applicable, add screenshots to help explain your problem. image

Version vmagent v1.55.1

Spec

    containers:
    - name: vmagent
      readinessProbe:
        initialDelaySeconds: 60
    extraArgs:
      promscrape.maxScrapeSize: 100MB
      promscrape.suppressDuplicateScrapeTargetErrors: "true"
      promscrape.suppressScrapeErrors: "true"
    image:
      repository: victoriametrics/vmagent
      tag: v1.55.1
    podScrapeNamespaceSelector: {}
    podScrapeSelector: {}
    relabelConfig:
      key: global_relabel.yaml
      name: monitoring-vmagent-cluster-relabel-config
    remoteWrite:
    - url: http://vminsert-monitoring-vmcluster-persistent.victoria-metrics.svc.cluster.local:8480/insert/0/prometheus/api/v1/write
      urlRelabelConfig:
        key: cluster_specific_relabel.yaml
        name: monitoring-vmagent-cluster-relabel-config
    replicaCount: 1
    resources:
      limits:
        cpu: 2
        ephemeral-storage: 10Gi
        memory: 16Gi
    serviceAccountName: vmagent
    serviceScrapeNamespaceSelector: {}
    serviceScrapeSelector: {}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 23

Commits related to this issue

Most upvoted comments

It does seem to use lower memory usage @valyala. I will monitor further for any spikes. image

The commit 4b18a4f026b301344e7fafb3dfec745b54fee23e should reduce logging verbosity when reloading -promscrape.config with big number of scrape config jobs.