moby: Dockerfile: ADD does not honor USER: files always owned by root

@thaJeztah i tried to be ironic about the developers lamenting that fixing this bug would be a breaking change. no shit, who would have thought!

just the amount of discussion that happens in the numerous threads that i encountered is a lot of time wasted, both by the developers and by the users. and all that for what? trying to protect some hypothetical users who knowingly relied on the behavior that COPY, rather surprisingly, ignores USER?

what are major version numbers for? and release notes?

if there actually exists anyone who relies on this bug then they should not upgrade to a new major release, or read the release notes.

and let’s say their code is broken because they didn’t read the release notes… what about the tons of users who get bitten by this every single day? maybe one outweighs the other…

I’ve created a PR that implements this (again): #21088.

So, this is considered to be a breaking change while renaming the whole project is not?

Hi Team, Kindly advise what is the status of this issue , because it does not make sense having this unexpected behavior of ADD

Current behavior:

ADD source_1gb.file destination_directory

RUN chown -R 775 destination_directory

Expected: 1gb Actual:2gb

suggested solution:

ADD source_1gb.file destination_directory --USER=non_root

ADD source_1gb.file destination_directory --REQ_PERMISSION=755

just the way we have RUN command which follows below USER non_root RUN echo “hello i am runninga s non root”

or give us the option to run command “ADD” like “RUN” in 2 modes "root and non root "

i am planning to open new issue to have flexibility to run “ADD” command in 2 modes "root and non root "

Please fix the issue of ADD , its really expensive because its increasing size of image by adding extra layer

also we understand we can avoid this issue by maintaining proper permission of file or directory to be added , but that’s extra overhead and not user friendly

Thanks Ripunjay

Hey, after reading all the comments. I’m not sure whether there any of the solution has been merged. Still facing the same issue in docker version 1.12.3.

Anyone able to help here? The only problem is of the big image size in executing the chown command.

This behavior really, really needs to be changed to make sure ADD uses current USER. Current way of implementing it with chown:

1. Created 3 (sic!) unnecessary layers
2. Makes Dockerfile ugly
3. Is totally not intuitive.

Keep in mind we’re recommend that people run their services as a non-privileged user. Inconveniences like this behavior will prevent people from following that.

I just got bitten by this and gotta say it’s a surprise that ADD does not respect the preceding USER declaration. If someone wants to add as root, simply run ADD before the USER is set to something else, as one would do w/ RUN.

+1 for this

I thought I had to be misunderstanding something. Is there some story that’s not being told here? There’s no logic in having a USER command which continues to act as root!

Wow. I’m a complete Docker newb and just ran into this one. From my virgin perspective, I absolutely expected ADD and COPY to respect USER. That they don’t was a source of considerable astonishment that increased exponentially when I saw just how long ago this issue was raised and how simple the apparent fix is. I don’t generally post to zombie threads, but felt obliged to add my insignificant voice to the general chorus of bafflement expressed here.

how about a ADDUSER and COPYUSER commands which would do an implicit chown?

Proposal: a second form of the ADD command. I’d love to be able to do:

ADD ["filename", "owner", "group", "755"]

Well, just to vent my frustration - this just caught me out (and I’ve been using Docker for a year now). How is this still a problem?

To save those who are interested in seeing this fixed a lot of time: here is the current attempt to implement a fix.

As an alternative I’ve been looking at Rocker, which implements this feature, and I think it fits the bill perfectly. Rocker is a tool that builds Docker containers, extending the spec of Dockerfiles. Using it in a CI environment should be absolutely straight forward.

I too am intensely curious whether any sane solution was implemented. I can’t afford doubling the size of the stuff we copy in.

Out of curiosity, why was the idea of simply adding a COPYUSER and ADDUSER command rejected? Simply duplicates COPY and ADD but respects previous USER directives. No backwards compatibility issues, and should be fairly straightforward to implement and document. It doesn’t have the same flexibility as adding --user and --group flags, but it would cover 98% of use cases and be much safer to implement.

about backwards compatibility, just update the major version and call it a day.

As a workaround using Alpine for me it was:

• COPY xfile location/xfile
• RUN adduser -S xuser
• RUN chown xuser -R location/xfile
• USER xuser

I’m subscribed on this issue since maybe two years, and I’m really puzzled why no solution has been found, whatever this looks like

One of the first arguments was that the Dockerfile format is frozen. That might have been true at that time, but does not hold anymore as there as been some changes since then (like FROM … AS or adding HEALTHCHECK)

Since the last release the option --from has been added to COPY. So there is also no reason anymore against adding options to Dockerfile keywords.

So what are the arguments against a COPY --chown <uid>:<gid> these days?

Hey @jessfraz is the reason why this issue was closed still valid?

This was commited a few months ago: https://github.com/moby/moby/commit/07250457df03396ea61dbf3782b5f8cf1094e7e0. I’ll leave it just here.

@kiwenlau @ripun @david-mccullars This is to comment on the additional layer size introduced by the chown command. I tested a more drastic situation on my Ubuntu system (in Oracle VirtualBox) where more than 1 chown instructions are placed in the Dockerfile, and indeed the image size gets larger and larger with additional chown instructions in the Dockerfile. But strangely such issue does not occur on our Jenkins server, which is a RHEL.

Eventually I found out that this is because the storage driver used by the RHEL server is devicemapper, whereas in my Ubuntu system it’s aufs by default. I forcibly changed the storage driver in my Ubuntu system to devicemapper and the additional chown command does not introduce the new layer disk usage any more.

docker history image_size_chown

With devicemapper the output is IMAGE CREATED CREATED_BY SIZE
a87cb09fe91f About an hour ago /bin/sh -c #(nop) CMD [“/bin/sh” “-c” "/bin/ 0 B
e1f41b263f7c About an hour ago /bin/sh -c chown -R martian /home/martian/ 0 B
f3169ab0f1f5 About an hour ago /bin/sh -c chown -R martian /home/martian 0 B
e68d44c49296 About an hour ago /bin/sh -c mv /home/martian/test_images/som 59.77 MB

With aufs the output is IMAGE CREATED CREATED_BY SIZE
a87cb09fe91f About an hour ago /bin/sh -c #(nop) CMD [“/bin/sh” “-c” "/bin/ 0 B
e1f41b263f7c About an hour ago /bin/sh -c chown -R martian /home/martian/ 117.5 MB
f3169ab0f1f5 About an hour ago /bin/sh -c chown -R martian /home/martian 117.5 MB
e68d44c49296 About an hour ago /bin/sh -c mv /home/martian/test_images/som 59.77 MB

And the second image is exactly larger by 235 MB in size, introduced by the two chown commands.

Docker has documented on storage driver selection here.

The default storage driver selection can be checked via sudo docker info. To configure docker daemon to use a different storage driver, add below line to /etc/default/docker and restart docker service. Note that images created under the original storage driver will NOT be available under the new one. They can be seen again when the storage driver option is restored. DOCKER_OPTS="--storage-driver=devicemapper"

what if incase the files I’m copying inside the docker image is of 1Gb. I can’t afford to have my image size increased by 1GB.

It also may take an unacceptably long amount of time.

@italomaia Not sure, why this issue was closed?

@thaJeztah This is not fixed.

@ehernandez-xk Hey, Looks like you didn’t read the previous comments. Though this seems to be a workaround, but this is going to increase your image size by the size of your xfile.

what if incase the files I’m copying inside the docker image is of 1Gb. I can’t afford to have my image size increased by 1GB.

Heh, yeah. It’s just that there’s a very strong split personality regarding backwards compatibility issues. For example, my PR fixing this issue (which everyone agrees is better than the current setup) was rejected for not being “backwards compatible” – an argument I do not understand. On the other hand, even with API versioning, clients aren’t backwards compatible with old servers (something that has actually bit us at SUSE).

I wish we’d all agree on what does and doesn’t make sense to complain about.

Am i doing something complete the wrong way?

RUN useradd -d /home/hybris -u 1000 -m -s /bin/bash hybris
VOLUME /home/hybris
ADD hybris /home/hybris
RUN chown -R hybris /home/hybris

Still all files belong to root.

Being forced to ADD and then RUN chown has a very nasty side-effect of artificially bloating docker images. For example:

FROM busybox:latest

RUN adduser -D bob
ADD 30MB_file /30MB_file
RUN chown bob /30MB_file

I ran build on this once without the chown and once with:

➔ docker images | grep bob
test    with_chown          a27913861eca        6 days ago          65.35 MB
test    without_chown       7ab2bbe29b50        6 days ago          33.89 MB

And we can see what is happening via docker history:

 ➔ docker history test:with_chown
IMAGE         CREATED        CREATED BY                                      SIZE
a27913861eca  6 days ago     /bin/sh -c chown bob /30MB_file                 31.46 MB
7ab2bbe29b50  6 days ago     /bin/sh -c #(nop) ADD file:9474e987e88bd93248   31.46 MB
e9413ccecaa1  6 days ago     /bin/sh -c adduser -D bob                       3.955 kB
4986bf8c1536  2 weeks ago    /bin/sh -c #(nop) CMD [/bin/sh]                 0 B
ea13149945cb  2 weeks ago    /bin/sh -c #(nop) ADD file:8cf517d90fe79547c4   2.433 MB
df7546f9f060  3 months ago   /bin/sh -c #(nop) MAINTAINER Jérôme Petazzo     0 B
511136ea3c5a  19 months ago                                                  0 B

When we consider that a large application might have several hundred megabytes or more, this chown becomes very expensive.

baaaah, you must be kidding me, it’s several years now… fixing this would be a breaking change?! hordes of people get bitten and annoyed by this every single day (me included), and you are afraid of a breaking change? when the “feature” that would be “broken” is pretty much just a bug…

I too would like to see this issue resolved. The current behavior really does not make sense.

@thiagowfx Your tl;dr summarizes a little too aggressively - it bears mentioning that this will double the size of your docker image.

I’m not sure why it isn’t acceptable to just make a new directive, say COPY2 which behaves as expected and can be optionally used by people that care without breaking anyone using the unintuitive COPY. Everyone gets a solution, nothing breaks.

On 04/26/2017 09:20 PM, Sebastiaan van Stijn wrote:

@attila-lendvai https://github.com/attila-lendvai yes; fixing “bugs” can be a breaking change.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/moby/moby/issues/6119#issuecomment-297584736, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXeDrdOfjpjbpjTHOuaJHMGIAlo5PUYks5rz-1jgaJpZM4B_uID.

The additional build time is also a concern. For me at least, when adding a large archive, the chown -R seems to take longer than the original ADD operation.

On Jul 11, 2017, at 4:32 PM, Paul Phillips notifications@github.com wrote:

@thiagowfx https://github.com/thiagowfx Your tl;dr summarizes a little too aggressively - it bears mentioning that this will double the size of your docker image.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/moby/moby/issues/6119#issuecomment-314562581, or mute the thread https://github.com/notifications/unsubscribe-auth/AYDy45yLHw__ewUuVJez78OrPBT5dYJbks5sM9vggaJpZM4B_uID.

Perhaps there is some way to figure out how much storage space is being used on AWS to store docker FS layers that do nothing but change ownership of a previously-copied resource, and offer a bounty based upon the cost of such storage… Who knows, maybe Amazon would like a few terabytes of storage back…

On Jun 8, 2017, at 5:24 PM, David Antliff notifications@github.com wrote:

As with other issues, round and round we go. The fact that no Docker developer has reopened this speaks volumes about the way the Docker project is run. This is a common criticism of Docker’s developers and it’s a red flag for anyone thinking of building production infrastructure on Docker. I recommend looking at other options, as this situation doesn’t look like it’s going to improve.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/moby/moby/issues/6119#issuecomment-307231503, or mute the thread https://github.com/notifications/unsubscribe-auth/AYDy42P0kzpN4ALmiOSlagXjrfvm64JPks5sCGadgaJpZM4B_uID.

@attila-lendvai yes; fixing “bugs” can be a breaking change.

Well, not sure if doable together, but I would love to see this solved for COPY as well. On Mar 9, 2016 09:51, “Marc” notifications@github.com wrote:

That would be great, thanks!

— Reply to this email directly or view it on GitHub https://github.com/docker/docker/issues/6119#issuecomment-194187573.

There was, once, a suggestion to allow people to specify the user via a flag on the ADD/COPY command but that was rejected at all - despite it being backwards compatible (e.g. ADD --user=foo src tgt).

This is extremely painful when having to do an npm install with COPY followed by chown, and combined with the Mac filesystem performance issues! A simple COPY with chown would make a significant difference.

This issue was closed because … ?

Hello! We are no longer accepting patches to the Dockerfile syntax as you can read about here: https://github.com/docker/docker/blob/master/ROADMAP.md#22-dockerfile-syntax

Mainly:

Allowing the Builder to be implemented as a separate utility consuming the Engine’s API will open the door for many possibilities, such as offering alternate syntaxes or DSL for existing languages without cluttering the Engine’s codebase

Then from there, patches/features like this can be re-thought. Hope you can understand.

I think fix this problem will help to decrease images size. Specify users when ADD files is a great idea!

For example, if I ADD a binary like Hadoop, then chown it to hadoop user, the image size will increase 100+MB!

That’s why I decided to install hadoop under root user:(

This topic was discussed by the maintainers and, while they agreed that having USER specify the owner of files added by ADD / COPY would have been a better choice, changing this behavior now would be a breaking change, which would be too much of a risk.

For this reason, https://github.com/docker/docker/pull/9934 was created to come with alternatives. An implementation of this is created in https://github.com/docker/docker/pull/10775, which is currently under review.

For those interested, please follow the review/discussion on https://github.com/docker/docker/pull/9934 to check progress.