magento2: DI compilation crashes when composer classmap includes generated code
Preconditions (*)
Have the following in your project composer.json as recommended since #16435 was merged:
"psr-0": {
"": [
"app/code/",
"generated/code/"
]
},
Reproduced on Magento 2.3.1, but if those lines are in your composer.json file the issue will apply to many older versions as well since the relevant code hasn’t changed for a long time.
Steps to reproduce (*)
Run the following commands in sequence:
bin/magento setup:di:compilecomposer dump-autoload -obin/magento setup:di:compile
Note that this is a minimal sequence to reproduce the issue. Normally this would be seen with steps #1-2 run during one deploy and then step #3 run the next deploy.
Expected result (*)
Compilation succeeds both times.
Actual result (*)
Second compilation command fails with an error like this:
Warning: include(/var/www/html/vendor/composer/../../generated/code/Magento/Framework/App/ResourceConnection/Proxy.php):
failed to open stream: No such file or directory in /var/www/html/vendor/composer/ClassLoader.php on line 444
Analysis
The issue is that the core Cli module includes magic early behavior to clear out the environment when a compilation command is being run, including deleting the generated code directories (see method Cli::assertCompilerPreparation() and note it’s called before Cli::initObjectManager()). However, when the object manager is then initialized it tries to load classes from the Composer classmap that no longer exist, since they were in the deleted generated directories.
The method assertCompilerPreparation() has the following comment:
/**
* Temporary workaround until the compiler is able to clear the generation directory
* @todo remove after MAGETWO-44493 resolved
*/
I don’t know what MAGETWO-44493 is, but it appears that the DiCompileCommand class does indeed already clear the generation directory itself (futilely since it’s already cleared by that point), so the comment is at a minimum misleading.
Possible solutions:
- Refactor the magic deletions out of the generic
Cliclass and into the relevant command classes after the object manager is initialized. This would be ideal, but I assume part of the goal is to prevent the object manager from loading old generated code, so I’m not sure how feasible this is. - Leave the existing magic as it is and add additional magic to remove or replace the Composer autoloader when compiling to avoid trying to access deleted generated code, thereby skipping straight to dynamic generation.
Workaround
Until this is fixed the error can be avoided by manually deleting the generated directory contents and then triggering a temporary classmap that excludes generated (either by running composer install or a simple composer dump-autoload) before running DI compilation. (Credit for the workaround and helping along my investigation to jalogut/magento2-deployer-plus#29, which I came across when Googling this problem.)
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 11
- Comments: 28 (19 by maintainers)
Commits related to this issue
- Magento2: dump autoload before setup di compile - fix for magento 2.4 deployments https://github.com/magento/magento2/issues/23251 — committed to unailopez16/deployer by unailopezvdshop 4 years ago
- Magento 2 Zero downtime deployment (#2262) * Magento2: look for database migrations in order to get zero downtime deployments when changes are not needed - also keep maintenance status active in cas... — committed to deployphp/deployer by unailopez16 4 years ago
@lbajsarowicz: you can just remove
--optimize-autoloaderfrom yourcomposer installcommand. The autoloader will get optimised by yourdump-autoloadcommand a bit later.Hi @scottsb, we recommend the following deployment flow:
composer installbin/magento setup:di:compilecomposer dump-autoload -oWe are not going to add additional magic to handle the case without
composer installFurther to the discussion taking place here, I can confirm that the following works in 2.4.3 and makes a significant performance improvement. The autoloader is added to OpCache following this command, I haven’t tested it in earlier versions.
composer dump-autoload && bin/magento s:di:c && composer dump-autoload --optimize --no-dev --classmap-authoritativeI always wondered if anyone is doing Code Review of Adobe Core developer changes.
Let’s look at the “fixes” introduced in Magento 2.4.2: https://github.com/magento/magento2/commit/23c61bd49b5323d08f2cd4592ee451ca03e4b9e1
This is strictly related to the issue we are encountering (Look at the references to non-existing classes like
\Magento\Framework\App\ResourceConnectionConnection)@hostep When I first ran into this issue, it was not on a local environment: it was via our house-built deploy scripts that we use for staging and production deploys. As referenced in the issue, it also came up for M2 Deployer Plus, a reasonably popular open source set of deploy scripts. So while it’s true that you shouldn’t use an optimized autoloader locally, that’s also somewhat non sequitur.
As far as providing a “realistic” set of deploy commands, that seems to be to be rather pointless, as it all it would do is add noise to the actual relevant steps. I mentioned in the issue description that it is the minimal reproducible example. I will point out this from the description in case you had skimmed past it originally, as it is key to seeing how this could be a “realistic” scenario:
Note that unlike @PascalBrouwers we were not deploying on an empty directory. But unless it becomes an officially documented requirement to deploy only on an empty directory, that’s not sufficient reason to disregard this issue.
Yes, I am talking about a deployment starting from an empty directory.
I’m just describing how easy it is to reproduce this issue. I am not going to list the whole set of commands used to deploy. Just use capistrano, deployer, etc.
But as you’ve mentioned: “On a local installation you can indeed very quickly trigger the bug”. Also, @scottsb provided a very in-depth analysis of where this bug is coming from.
@scottsb the important part is “various problems when adding/removing classes.” and
setup:di:compileadd Factory, Proxy, etc classesI’m just writing here my experience regarding this.
In our websites without running
composer dump-autoload -o --apcuwe’re getting 1.5 seconds slower execution speeds due to file loading by composer.
By running just
composer dump-autoload -owe’re getting errors like above.
It seems in devdocs https://devdocs.magento.com/guides/v2.4/performance-best-practices/deployment-flow.html there’s some documentation on how to deploy composer optimized via using apcu cache, but it’s not really well visible and it doesn’t explain why it works only with apcu cache enabled.
I would expect not having to use apcu would result the same as using apcu
Now the only working order for us is after installing dependencies we execute deployment commands in this order
Any other attempt with optimized autoloader breaks everything for us.
@hostep Looks like it works (however, I tried that before and for some reason, it didn’t work). Thank you.
I’m not seeing any error in your output. Is there a specific error? Are you sure you are running into the issue this ticket is about?
You should get an error like this
Class Magento\Framework\App\ResourceConnection\Proxy does not exist(Magento 2.3.6) orWarning: include(vendor/composer/../../generated/code/Magento/Framework/App/ResourceConnection/Proxy.php): failed to open stream: No such file or directory in vendor/composer/ClassLoader.php on line 444(Magento 2.4.1)You can always fix this issue by just running
composer install(so without an optimised autoloader)If this isn’t the issue you are running against, I suggest you search help elsewhere 🙂