composer: Composer is not working in vagrant synced folders of a virtual machine

According to @rpkamp I used the workaround but had to change the script a little for Laravel Homestead.

I had to create a script /usr/local/bin/unzip with such content:

#!/bin/sh

/usr/bin/unzip "$@"
sleep 0.2

And remember you have to give it chmod +x /usr/local/bin/unzip.

Create a file /usr/local/bin/zip in the VM with the following content:

#!/bin/sh

/usr/bin/unzip "\$@"
sleep 0.2

It hijacks zip on your system so that every time composer executes zip it will execute this script rather than the actual zip program. This script will then call the actual zip program and then wait for 200ms after zip finished so that hopefully in that time changes have been really synced to disk. You can of course play with the 0.2 to see if any other values works better. 0.1 did not work for me, but that probably also depends on disk speed (I do have an NVMe SSD, but it’s a few years old, so I’m not sure how that would stack up against a more modern SSD).

@LanmanGroup thanks! Sorry I completely messed that up and forgot the install step for plugins, only did it the sleep on update. Anyway please try again with latest composer self-update --snapshot, if that really doesn’t fix it then maybe I’ll give up.

Ok it should detect vagrant username or COMPOSER_RUNTIME_ENV env var set to virtualbox, and do a few sleeps that are hopefully strategically-enough placed that they fix the issue without causing too much slowdown.

Please try it with composer self-update --snapshot and let me know if it pans out.

@tomaszstrojny’s solution above worked for me (Laravel Homestead project, php7.4, virtualbox and vagrant)

Here it is summarized with more details, for those who need it:

1. Create a script named unzip, somewhere. May as well be in the root of your project (where composer.json lives), with:

#!/bin/sh

/usr/bin/unzip "$@"
sleep 0.2

1. Enter the Vagrant instance: vagrant ssh

2. Copy the script to usr/local/bin (of the Vagrant instance): sudo cp /path/to/unzip /usr/local/bin

3. Allow the file to be executeable, with: sudo chmod +x /usr/local/bin/unzip

4. Try running composer install again, in your site root (still in vagrant ssh)

So the problem is Composer! Or not?

@webprogrammierer put your vendor folder outside the virtualbox shared folders. That’s the workaround I know of (I’m not using virtualbox myself).

I don’t think composer can do anything here. That’s an issue coming from the virtualbox shared folder filesystem, where files are reported as non-existent in PHP filesystem APIs. This issue has been reported to virtualbox since years AFAIK but does not seem to have been fixed. And the faster composer becomes, the more likely it is to trigger this virtualbox issue (which seem to appear when trying to read from the filesystem too soon after writing to it).

@hochleitner https://github.com/composer/composer/commit/8427b6c8ed335a09a7b03e8ee45bd12c788135ad hopefully fixes it for the binaries too. Can you check with composer self-update --snapshot?

My gut says my old issue was also somehow related to my old issue which was about running >=15 (Drupal) Composer installations on a dedicated AWS instance in parallel. So probably this issue is not about Vagrant or Virtualbox, it is more like about parallelism and filesystems. We tried to tweak inode size on the AWS instance but that did not help. The only currently working solution is not sharing Composer cache between those parallel builds, which increases network traffic and kinda eliminates the purpose of Composer package caching. I wonder if this fix would also solve that problem, I may give it once. https://github.com/composer/composer/issues/9568

A workaround and not a solution, but in case anyone reads up to here, and still has problems with VirtualBox shared-filesystem being both slow and unreliable, for apps other than composer:

my goto solution for running php-on-linux on a windows host has been for years to

• run linux in virtualbox (or vmware)
• run php in linux, including composer
• do not use vbox (nor vmware) native file-shares
• do not share the host disk to vbox, but do the opposite: share the linux disk’s www root to windows via samba

pros:

• never have any issue with fielsystem casing (windows not liking hello.txt and Hello.txt in same folder)
• all linux things run at native speeds
• samba share perfs on the windows side are “quite good” for most operations. Never had caching issues with file metadata
• allows to encrypt your data using vbox encription, no need for bitlocker or veracrypt

cons:

• need to be able to install and configure samba
• php code is not accessible without the vm running
• size of vm disk might spiral out of control
• running tortoisegit on windows to manipulate git repos in the samba shares works, but it is much slower than git cli in linux. Also there might be locking issues if the two gits are run at the same time
• running phpstorm in windows is suboptimal as a) it will not refresh files instantaneously on disk changes, nor b) have access to all the cli tools running in the vm

Please beware: COMPOSER_RUNTIME_ENV now expects virtualbox instead of vagrant. But I also added support for lsmod to detect vbox guest additions so it should hopefully not be needed at all to set the environment variable.

All this is now part of Composer 2.0.12 which was just released.

@Gamblt you can try to run composer with COMPOSER_RUNTIME_ENV=vagrant set in the env, like COMPOSER_RUNTIME_ENV=vagrant composer install should do. I am not fully convinced it’s the same issue though as it’s another environment and another error message. Maybe best report this in a new issue and include all the output we request in the issue template.

@enursed that’s an unrelated issue, and is due to filesystem performance, to which frankly the only solution is getting rid of your slow filesystem. Please do not side-track this issue.

@LanmanGroup @inverse great to hear the vagrant/vbox issue appears fixed!

@viliusle trying out if my fix helps would be more helpful than asking me if it’s fixed 😉 I don’t have vagrant, I don’t know. I’m just trying to help here.

@webprogrammierer no. the issue is with the shared filesystem of the VM (IIRC, Docker for Windows relies on a VM running Linux and so involves such shared folder).

@phpguru try to write a simple PHP script using \FilesystemIterator to iterate over the files in your big folders and count them. I suspect that you might face the same issue than the one faced by composer (but maybe there’s a timing condition involved, as it is not clear whether some files are never synced to the VM or whether they only have a delay before appearing)