cli: Docker log rotation is not working the container log keeps on growing

I have an issue that even though I set the docker log configuration it does not kick in and my docker logs grows till I run out of disk space. Can you please tell me when the log rotation is cleaned up, I have the following config set for my container when I run

docker inspect <container-id>

      `"LogConfig": {
            "Type": "json-file",
            "Config": {
                "max-file": "5",
                "max-size": "100m"
            }
        }`

and my log size is above 400 MB right now, but the log rotation hasn’t kicked in.

Steps to reproduce the issue:

docker run --name badguy --log-opt max-size=1k --log-opt max-file=5 -d ubuntu sh -c “while true; do printf A; done”
wait for a while
docker kill badgu

Describe the results you received: my log size is above 1k right now, but the log rotation hasn’t kicked in.

Describe the results you expected: Log rotation with file size not exceeding 1k

Output of docker version:

Output of docker info:

Containers: 39
 Running: 1
 Paused: 0
 Stopped: 38
Images: 1441
Server Version: 17.03.1-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 1214
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.13-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952 GiB
Name: moby
ID: 6GE7:EIWS:LOD2:IFS2:3UHV:A62R:FWGU:RYUM:LTSJ:7D4B:VWES:DLEL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 23
 Goroutines: 32
 System Time: 2018-06-26T10:22:08.783714835Z
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 3
Comments: 43 (6 by maintainers)

Most upvoted comments

You cannot change the configuration for existing containers; when you mention “expanding the logs”; were you trying to change the configuration after containers were already started with the old configuration? Are your tools attempting to change/update the configuration of existing containers?

thaJeztah on Jul 2, 2018

I can confirm that you have to recreate the container to make existing containers work. This should be added to the docs.

docker version: Docker version 19.03.12, build 48a66213fe

yifeikong on Nov 18, 2020

Nevermind. I’m stupid and wasn’t fast enough to delete/revise my comment here. Sorry to have bothered you. Hope you are healthy and doing fine!

MondaySam on May 19, 2020

I do not think there is any reason to try and make this so exact. There is a 1MB line buffer. We always write a full line (plus metadata like timestamp and which i/o stream it came from) to the log.

Having a log file stay smaller than 1MB and also write lines as large as 1MB does not seem like a real use-case. Likewise having a container write lines that are larger than the max log file size does not seem like a real use case.

I’m going to close this as issues seem to be related to incorrect configuration. We have tests for this in tree as well. If you are having this issue on Docker 19.03.6 please let us know the details. And remember, containers must be re-created if you are relying on changes made daemon.json. Thanks! 😇

cpuguy83 on Mar 5, 2020

Was there a solution to this. I am facing the same problem

Yes. You need to recreate the container. Docker restart does not suffice here. https://github.com/docker/cli/issues/1148#issuecomment-645990725 + https://github.com/docker/cli/issues/1148#issuecomment-729539456

MondaySam on May 20, 2021

The solution to my problem was rather simple. Docker restart is simply not enough. If the container was already running after you added the log rotation, you must use docker up -d again.

MondaySam on Jun 18, 2020

This looks like compose is not passing down the options for some reason. I’m not intimately familiar with the compose schema here.

Doing a docker run manually with log options works just fine.

cpuguy83 on Nov 5, 2019

Any update on this issue?

miande-acxiom on Feb 21, 2019