influxdb: High memory usage

Hi, I install Influxdb on ubuntu server with 4G of memory and use python’s requests module to write 10M points into db. The python script inserts 20k points per second into db successfully but ubuntu’s memory goes high until Influxdb use 97% of Ram. After that I can’t query a simple request like “select * from srcipm limit 1”. Even after write process is finished, Influxdb doesn’t release memory.

Details:

Server: 64bit Ubuntu Server 15.10, 1 Core CPU, 4G RAM
InfluxDB Version 0.9.6

Write query in python:

requests.post("http://192.168.1.104:8086/write?db=mydb", "srcipm,device_id=4,input_snmp=2,output_snmp=3,direction=1,ip=192.168.1.1 bytes=25478")

Where is the problem? should I config something or my query is wrong?

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 38 (14 by maintainers)

Commits related to this issue

Update influxdb.yml picked some suggestions from https://github.com/influxdata/influxdb/issues/5440 to try to reduce high memory usage from influxdb — committed to usegalaxy-eu/infrastructure-playbook by gmauro 4 years ago

Most upvoted comments

@carbolymer It looks like you have sparse data (stock prices) which ends up creating hundreds of small shards. In your docker sample, I’d recommend increasing the shard group duration on your default retention policy after creating the databases.

For example, running the following before writing data:

alter retention policy default on no_memory shard group duration 520w

will change the shard group duration to 10y which should reduce the number of shards from ~1500 to 4.

I would also suggest setting cache-snapshot-write-cold-duration = "10s". You should not need to change compact-full-write-cold-duration or cache-snapshot-memory-size from the default values though.

jwilder on Jul 18, 2016

Try reduce cache-snapshot-memory-size and cache-snapshot-write-cold-duration in the configuration. I set mine cache-snapshot-memory-size = 2621440 and cache-snapshot-write-cold-duration = “1m” during massive writes.

lpc921 on Feb 25, 2016

I’m having the exact issues as everyone here. During inserts my memory usage jumps over 8G and then influx throws error about memory allocation failure (my VM is limited to ~8GB RAM). I’ve tried settings proposed by @lpc921 in his post here:

  cache-snapshot-memory-size = 262144
  cache-snapshot-write-cold-duration = "0h1m0s"
  compact-full-write-cold-duration = "0h1m0s"

It didn’t change a thing.
I am using your official docker image: influxdb:0.13-alpine

Guys, seriously. This issue has been open for half a year. For me this is a critical issue which rules out usage of influx on a production environments.

EDIT: I’ve prepared docker containers which demonstrate this bug in influxdb 0.13. You can find them here: https://github.com/carbolymer/influxdb-large-memory-proof EDIT2: The same happens for the 1.0.0-beta2-alpine image. EDIT3: Many thanks to @jwilder for the advice! The latest commit on master: https://github.com/carbolymer/influxdb-large-memory-proof contains working solution.

carbolymer on Aug 4, 2016