symfony: Autowiring is very slow for large excluded directories

Q A
Bug report? yes
Feature request? no
BC Break report? no
RFC? no
Symfony version 3.4.*

I’m experiencing a significant slowdown from autowiring when there are large directories in a bundle (e.g. node_modules), even when those directories are excluded.

This configuration is slow:

services:
    _defaults:
        autowire: true
        autoconfigure: true
        public: false

    AppBundle\:
        resource: '../../src/AppBundle/*'
        exclude: '../../src/AppBundle/{Entity,Resources,Repository,Tests,node_modules}'

This one isn’t, but has the downside of adding each directory to resource manually.

services:
    _defaults:
        autowire: true
        autoconfigure: true
        public: false

    AppBundle\:
        resource: '../../src/AppBundle/{Publisher,Locator,Foo,Bar}'

My poorly educated guess is that the GlobResource in the config component iterates through all the possibilities first, before they are filtered out by the exclude pattern? This would explain the slowdown.

https://github.com/symfony/symfony/blob/b3fc3b569003ea9454449391a01d516d0a560f9a/src/Symfony/Component/Config/Resource/GlobResource.php#L97

Do you think it would be possible to prevent the glob loader searching through excluded directories?

The problem is so bad that I occasionally get out of memory errors in dev:

[Sun Apr 01 19:24:47.554299 2018] [:error] [pid 4298] [client 192.168.10.1:62972] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 130968 bytes) in /var/www/project/vendor/symfony/config/Resource/GlobResource.php on line 105

Thanks very much for adding autowiring to Symfony. It has made development so much faster! 🎉

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 19 (12 by maintainers)

Commits related to this issue

Most upvoted comments

Thank you @gonzalovilaseca and @nicolas-grekas for your hard work on this! I really appreciate the effort.

@glynnforrest Wonderful, thanks, will try to add tests and do a proper PR now.

@gonzalovilaseca Thanks for your patience, sorry for my late response.

I ran the toolbox test script with your patches and it doesn’t seem to be looping through the src/NoLoad folder anymore, nice!

I can’t speak for the impact of this patch elsewhere, but I’m very happy!

screen shot 2018-07-14 at 23 58 41

There is a mistake I made when merging as nicolas pointed out. Let me fix it and I’ll get back to you.

Please continue 😃

Could be useful (although not directly related to the described issue): https://yarnpkg.com/lang/en/docs/workspaces/