moby: File mount does not update with changes from host

I bet I know what’s happening here…

If you are using some editor like vim, when you save the file it does not save the file directly, rather it creates a new file and copies it into place. This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container. When the container is restarted the new inode. If you edit the file in place you should see changes propagate.

This is a known limitation of file-mounts and is not fixable.

Does this accurately describe the issue?

I made some tests @greg0ire, and this is the result.

With or without set noswapfile the inode changes, everytime the file is saved, but I made a little research how to make vim don’t change the inode of a file. The result was to use set backupcopy=yes, I try and the inode don’t change when the file is modified.

Thank you guys.

Closing, it’s not really a bug, just how linux works unfortunately.

@OscarOrSomething Mount a dir instead of a file.

Turns out it was the opcache settings in the wordpress image I was using. Adding the below lines to my Dockerfile fixed the problem for me (no problems in an hour). Now the files are showing in the container and when previewing with apache.

FROM wordpress:latest
RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini

After reading this, I guess if I type set noswapfile, I might be able to continue using vim. Maybe Sublime Text has a similar option ?

@cpuguy83 I am mounting a dir.

After some googling, Sublime text has atomic_save instead so Adding "atomic_save": false to user preferences worked (After a restart). Thanks @greg0ire

Turning off sendfile off for Nginx solved this issue for me. Just put this in your vhost conf:

sendfile off;

The equivalent for Apache it is “EnableSendfile off”. See https://abitwiser.wordpress.com/2011/02/24/virtualbox-hates-sendfile/ for reference.

have the same problem

volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d

i change for this and it work

volumes:
      - /work/docker/nginx/conf.d:/etc/nginx/conf.d

i think the reason is

./

These are not issues with Docker. Mounting individual files is tricky. A file path and a file inode are different things.

A file path is more akin to a git tag where as an inode is like a commit hash. Just like a git tag can be changed to point to a different hash, so can a file path point to a new inode. A commit hash is definite, it doesn’t change, likewise for an inode. So, file paths are just references to an inode. These references can and do change depending on the actions you perform (e.g. mv /foo /bar defeferences /foo from the underlying inode and creates a new reference /bar to that same inode)

When you bind mount a path (file or dir), you are mounting an inode to a new location (kind of like creating a new git tag pointing to the same commit hash).

If you do something like move a bind-mounted dir to a new location, and then replace it with another dir, the bind-mount will follow the original dir, same for files. With many editors, the editor will write to a temp file, then replace the real file with the temp file on save, in effect dereferencing the inode (like pointing a git tag to a different commit) from the given path and a new inode taking the reference. Since mounts follow the inode, not the path reference, on the host side you are reading/writing to a new inode, on the container side you are reading/writing from the original inode.

It’s how the linux kernel works 🤷‍♂️ you can try it on your host (on a linux machine, but probably works on macOS as well);

# create your file on the host
echo "# my-file" > myfile.txt

# create a directory (this is the "container's filesystem")
mkdir container

# create the mountpoint inside the container's filesystem
touch container/myfile.txt

# mount the file from the host into the container
mount -o ro,bind myfile.txt container/myfile.txt

# read the bind-mounted file "inside the container"
cat container/myfile.txt
# my-file

# "modify" the file, in this case, using `sed`, which uses a temporary file, then replaces the file
sed -i 's/my-file/my-file-updated/g' myfile.txt

# check that the file "on the host" is updated
cat myfile.txt
# my-file-updated

# check the mounted file "inside the container:
cat container/myfile.txt
# my-file

# unmount the file
umount container/myfile.txt

Thanks to @greg0ire and @cpuguy83 . It’s always a relief to me when some mysterious technical glitch that happened to me has a decent reason.

Unfortunately I don’t think there’s a way to make bind-mounting single files work; mounting a folder is really the only solution if the editor does a “move old, and write updated file to old location”

@cpuguy83:

1. displaying a warning does not require kernel support
2. monitoring how many path points to an inode is already supported. So overwriting a file with a new one is detectable.

docker run ... -d /home/user/data/something.txt:/data/something.txt echo "new content" > /home/user/data/something2.txt mv something2.txt something.txt

Then the inode table have one row less, so it is detectable. Docker already knows the original path (/home/user/data/something.txt), also knows the inside path (/var/lib/docker/volume/.../something.txt), so he can check and remount the new path.

Or file path in volumes should be disabled. It is more trouble than good.

This thread is amazing. I was running into this problem and effectively, yes, I was using vim and yes, set backupcopy=yes makes things work as hoped. Thanks for resolving my problem users of github 😊

Ouch. So this is why we sometimes have to reload our containers inexplicably. This is a dreadful bug, I must say that I don’t really agree with it being closed. When explicitly volume binding a file name I would expect changes to the file with that name to be reflected in the container without the need to have knowledge about the intricacies of the file system.

Is it really impossible to monitor for example the modified time stamp and update the inode reference accordingly?

My workaround was to ln /tmp/foo path/to/foo the extra reference count on the inode means it doesn’t change when the editor writes the new file.

I had the same problem on Windows 10 after updating Docker to 2.1.* To solve it I rearrange my app structure.

volumes:
- './:/var/www/html/app/'
- './protected/data/mysql:/var/lib/mysql'
- './docker/php/vhost.conf:/etc/apache2/sites-enabled/000-default.conf'
- './docker/php/cert/:/etc/apache2/ssl/'
- './docker/php/config/php.ini:/usr/local/etc/php.ini'
etc ...

I think mapping whole directory ./ in combination with other mappings caused problem.

Yesterday I faced this issue when using Docker HostManager - /etc/hosts wasn’t updated though DHM container was working. Yes, I use vim for simple editing and didn’t have idea that swapping files on save causes inode change and Docker file mount desynchronization. I’ll sum up what I did:

• Updated ~/.vimrc with:

  " Disable swapping files in the same directory as edited file,
  " mostly for editing /etc/hosts which is mounted to Docker HostManager
  " and swapping causes inode change and breaks DHM.
  " @see https://stackoverflow.com/a/15317146/842480
  " IMPORTANT: directories must exist, so create them!
  :set backupdir=~/.vim/backup//
  :set directory=~/.vim/swap//
  :set undodir=~/.vim/undo//
  

• NeoVim users should also add source ~/.vimrc to ~/.config/nvim/init.vim
• Added ACL to /etc/hosts so I don’t need to use sudo and my own .vimrc is loaded: sudo setfacl -m u:your-user:rwX /etc/hosts

Now, when I edit /etc/hosts manually, it is instantly propagated to DHM container and everything works like it should. Thanks all for your comments, it helped me to find real issue 😃

References:

• https://stackoverflow.com/a/15317146/842480
• https://stackoverflow.com/questions/3495124/not-reading-vimrc#comment86939843_28422013

If someone is facing the same issue with Docker PHP image, another helpful thing is to disable any PHP cache extension e.g. opcache. memcache, etc

I bet I know what’s happening here…

If you are using some editor like vim, when you save the file it does not save the file directly, rather it creates a new file and copies it into place. This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container. When the container is restarted the new inode. If you edit the file in place you should see changes propagate.

This is a known limitation of file-mounts and is not fixable.

Does this accurately describe the issue?

Dazed and confused… saved my afternoon

I bet I know what’s happening here…

If you are using some editor like vim, when you save the file it does not save the file directly, rather it creates a new file and copies it into place. This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container. When the container is restarted the new inode. If you edit the file in place you should see changes propagate.

This is a known limitation of file-mounts and is not fixable.

Does this accurately describe the issue?

Thank you! My Volume Setup was the follow:

services:
  db:
    . . .
    volumes:
      - ./docker/db/startup.sh:/docker-entrypoint-initdb.d/startup.sh

My goal was to debug the script, after the container was created

so I changed the script and I executed it inside the docker container. nothing “new” happend.

I moved the startup script to a “startup” folder and changed the docker-compose.yml

services:
  db:
    . . .
    volumes:
      - ./docker/db/startup/:/docker-entrypoint-initdb.d/

I executed the script again inside the docker container and my changes w.ere recognized.

PS: my english sucks 😄

@greg0ire I try use set noswapfile, but nothing change… Isn’t possible use vim with volumes?

Is there any better solutions… I’m using VSCode facing same issue. Mounting folder instead of a single file is not a good idea.

@klaslo docker does not monitor anything.in this regard. It relies on the kernel, and the kernel does not support this. The right place to “fix” this is the kernel. We could introduce hacks, but I’m not sure it’s worth it, at least not right now.

I have Docker for Mac. Adding a new file with code works instantly. Yay, right? When I modify the file with Atom text editor, the changes take around a minute.

The image I’m using is the official Wordpress image, even if Docker for Mac fixes the inode stuff, it wouldn’t seem like it. Why? Because there seems to be a caching of files as @moneal pointed out earlier.

For example make a new file:

// my-plugins/stopper.php
<?php
  die('edit me 1');
// This will happen instantly when you reload

// my-plugins/stopper.php
<?php
  die('edit me 2');
// This will not instantly change, we changed it to number 2

// my-plugins/stopper.php
<?php
  opcache_reset();
  die('edit me 3');
// This will work instantly, we changed it to number 3

---

I’d hate to run the following, removing some intended functionality for production servers, is there a work around for local development, a hook or something that only runs locally.

RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini

---

I ended making a file in mu-plugins/_.php with the following code

<?php
  opcache_reset();

File modifications seem to be appearing instantly from what I can tell. This is probably not efficient or desired, but I’ll just end up adding it to the git-ignore. Hope this helps.

k8s isn’t fully compatible with Docker 1.11 yet, also k8s does some things outside of the “api” for Docker (direct access to files in /var/lib/docker, working around standard networking in docker), which is, well, not always the most reliable approach.

@OscarOrSomething I suspect that’s because VirtualBox shared folders don’t support inotify; https://www.virtualbox.org/ticket/10660

If you edit the file in place you should see changes propagate.

I just restarted my container, used nano to do the editing and it works! Thanks a lot. Feel free to close this, I am not sure if it should be considered as a bug or not.