containerpilot: memory leak
I was using ContainerPilot v3.3.0 and found the ContainerPilot process memory leak.
Linux 192-168-161-100 4.4.0-79-generic #100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux ubuntu 16.04 docker image: phusion/baseimage:0.9.18
ContainerPilot configuration:
{
consul: "127.0.0.1:8500",
jobs: [
{
name: "p-job",
port: 80,
exec: "p-job start",
restarts: "unlimited",
when: {
source: "generate-file",
once: "exitSuccess",
},
health: {
exec: "curl 127.0.0.1:80",
interval: 2,
ttl: 10,
timeout: "10s"
},
},
{
name: "consul-client",
exec: [
"consul",
"agent",
"-config-dir=/etc/consulClient.conf",
"-data-dir=/var/lib/consul",
"-retry-join=consul-server",
"-disable-host-node-id"
],
health: {
exec: "consul operator raft -list-peers | grep leader",
interval: 2,
ttl: 10,
timeout: "10s"
},
restarts: "unlimited"
},
{
name: "generate-file",
exec: [
"consul-template",
"-once",
"-consul-addr",
"127.0.0.1:8500",
"-template",
"/etc/containerpilot/servicesTemplate.ctmpl:/etc/containerpilot/services.json"
],
when: {
source: "consul-client",
once: "healthy"
}
},
{
name: "consul-leave",
exec: "consul leave",
when: {
source: "consul-client",
once: "stopping"
}
},
{
name: "regenerate-file-a",
exec: [
"consul-template",
"-once",
"-consul-addr",
"127.0.0.1:8500",
"-template",
"/etc/containerpilot/servicesTemplate.ctmpl:/etc/containerpilot/services.json"
],
when: {
source: "watch.a",
each: "changed"
}
},
{
name: "regenerate-file-b",
exec: [
"consul-template",
"-once",
"-consul-addr",
"127.0.0.1:8500",
"-template",
"/etc/containerpilot/servicesTemplate.ctmpl:/etc/containerpilot/services.json"
],
when: {
source: "watch.b",
each: "changed"
}
},
{
name: "regenerate-file-c",
exec: [
"consul-template",
"-once",
"-consul-addr",
"127.0.0.1:8500",
"-template",
"/etc/containerpilot/servicesTemplate.ctmpl:/etc/containerpilot/services.json"
],
when: {
source: "watch.c",
each: "changed"
}
},
{
name: "send-sighup-a",
exec: "kill -HUP ${p-job PID}",
when: {
source: "regenerate-file-a",
each: "exitSuccess"
}
},
{
name: "send-sighup-b",
exec: "kill -HUP ${p-job PID}",
when: {
source: "regenerate-file-b",
each: "exitSuccess"
}
},
{
name: "send-sighup-c",
exec: "kill -HUP ${p-job PID}",
when: {
source: "regenerate-file-c",
each: "exitSuccess"
}
},
],
watches: [
{
name: "a",
interval: 1
},
{
name: "b",
interval: 1
},
{
name: "c",
interval: 1
},
],
control: {
socket: "/var/run/containerpilot.socket"
}
}
pmap -d ${containerpilot PID}
Address Kbytes Mode Offset Device Mapping
0000000000400000 3788 r-x-- 0000000000000000 000:000bc containerpilot
00000000007b3000 2868 r---- 00000000003b3000 000:000bc containerpilot
0000000000a80000 216 rw--- 0000000000680000 000:000bc containerpilot
0000000000ab6000 140 rw--- 0000000000000000 000:00000 [ anon ]
000000c000000000 56 rw--- 0000000000000000 000:00000 [ anon ]
000000c41fe58000 1696 rw--- 0000000000000000 000:00000 [ anon ]
000000c420000000 7168 rw--- 0000000000000000 000:00000 [ anon ]
000000c420700000 3072 rw--- 0000000000000000 000:00000 [ anon ]
000000c420a00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c420b00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c420c00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c420d00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c420e00000 2048 rw--- 0000000000000000 000:00000 [ anon ]
000000c421000000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421100000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421200000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421300000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421400000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421500000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421600000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421700000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421800000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421900000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421a00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421b00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421c00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421d00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421e00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c421f00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422000000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422100000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422200000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422300000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422400000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422500000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422600000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422700000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422800000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422900000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422a00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422b00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422c00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422d00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422e00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c422f00000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c423000000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c423100000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c423200000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c423300000 1024 rw--- 0000000000000000 000:00000 [ anon ]
000000c423400000 1024 rw--- 0000000000000000 000:00000 [ anon ]
00007fa8b9817000 3584 rw--- 0000000000000000 000:00000 [ anon ]
00007ffee4d07000 136 rw--- 0000000000000000 000:00000 [ stack ]
00007ffee4da6000 8 r---- 0000000000000000 000:00000 [ anon ]
00007ffee4da8000 8 r-x-- 0000000000000000 000:00000 [ anon ]
ffffffffff600000 4 r-x-- 0000000000000000 000:00000 [ anon ]
mapped: 66776K writeable/private: 60100K shared: 0K
dump and view ContainerPilot memory, the memory stores a large number of duplicate the following:
- container environment variable
- a, b, c services registered to consul information
- Send-sighup event
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 37 (32 by maintainers)
We can confirm this bug has been fixed, no obvious leak after 16 hours running, thanks!
Sorry for the delay, @JeromeGe is taking a vacation right now, so Iām arranging anther colleague to provide a reproduce demo, will submit today, thanks for your help š