rector: [ERROR] Rector timeout after upgrading it to 0.19.0

Bug Report

Subject Details
Rector version 0.19.0

Since the upgrade of rector (from 0.18.13 to 0.19.0) I have this big error : image

I also try to folow these intructions without sucess https://getrector.com/documentation/troubleshooting-parallel

Minimal PHP Code Causing Issue

  1. Edit composer.json file
{
    "require-dev": {
        "rector/rector": "^0.19.0"
    }
}

  1. Run composer update

This is the 1st time Iโ€™ve had this at an rector update.

Expected Behaviour

Job passing like in version 0.18.13, just before upgrading rector to the 0.19.0

About this issue

  • Original URL
  • State: closed
  • Created 6 months ago
  • Comments: 36 (19 by maintainers)

Commits related to this issue

Most upvoted comments

have same problem here with phpstan/phpstan (1.10.57) and rector/rector (0.19.5). Timeouts with parallel,Up to 4x slow without parallel

@vinceAmstoutz See rectorphp/rector-src#5477

Let me know if we can improve the feedback ๐Ÿ‘

Thank you so much for your input. It help us to improve Rector further and fix the glitch to improve DX for other to come ๐Ÿ™

@TomasVotruba No, itโ€™s perfect! Thank you so much for all you do for the PHP community ๐Ÿ™๐Ÿ˜

@vinceAmstoutz See https://github.com/rectorphp/rector-src/pull/5477

Let me know if we can improve the feedback ๐Ÿ‘

Thank you so much for your input. It help us to improve Rector further and fix the glitch to improve DX for other to come ๐Ÿ™

@vinceAmstoutz Thanks for more debug info. Indeed itโ€™s not best practise to use UP_TO_* sets in rector.php longer than is their single-run. They run dozens of rules with very complex logic, as it take to get from Symfony 2 to 7. Itโ€™s like running a Tesla control and checking all elements available since the first horse ride ๐Ÿ˜ƒ

See section 3: https://getrector.com/blog/5-common-mistakes-in-rector-config-and-how-to-avoid-them

I recommend to run those just once and remove those ๐Ÿ‘

We got similar feedback from various project, seeing the Symfony โ€œlevel up toโ€ approach is not the best, neither practical to use. Iโ€™ll think about how to deprecate those to avoid similar hustle for other users.

Thank you

Thank you, that seems on specific symfony rules, see changes on latest rector-symfony diffs

https://github.com/rectorphp/rector-symfony/commits/main/

if you found something fishy, feel free to provide a patch ๐Ÿ˜‰

Thank you, at least it faster, I will merge the PR.

I will looking on other area that can improve performance.

Looking at the setlist you have, it seems you have huge of collection of setlist, could you narrow down and possibly found a rules that have bottleneck? Otherwise, we may need a reproducible repo.

@vinceAmstoutz thank you, thatโ€™s seems strange, I create a PR to add false default autoload to disable autoload on class_alias at:

for deprecating Rector\Core subnamespace.

Could you try copy src/core_namespace_aliases.php that uses false default autoload at:

https://raw.githubusercontent.com/rectorphp/rector-src/add-false-autoload/src/core_namespace_aliases.php

to your vendor/rector/rector/src/core_namespace_aliases.php and verify if that increase the performance?

@samsonasik So for now, Iโ€™ve got these benchmarks to share with you before testing @TomasVotruba suggestions

Disable Parallel Parallel (60, 30, 20) Parallel (120, 16, 15)
0.18.13 3min 8s 21ms 1min 15s 523ms 1min 2s 274ms
0.19.0 16min 54s 65ms 3 min 1s 202ms -> timeout 4 min 3s 717ms -> timeout
dev-main 14min 25s 850ms 2 min 56s 362ms -> timeout 2min 50s 267ms -> timeout

I also have screenshots of all the measurements I took if you need them.

Thank you, as short, disableParallel() must be compared with same disableParallel() config, and parallel() need to be compared with parallel() config.

I am not sure if I have time to call on this, it is better to have reproducible repo ๐Ÿ˜ƒ

@vinceAmstoutz Thanks for reaching out. We need to find out what commit caused this.

Could you give this approach a try and see which commit made the performance drop? https://github.com/rectorphp/rector/issues/8403#issuecomment-1893430494

On same disableParallel(), that seems impossible you get 1 minutes as well, while 0.19.x you got 20 minutes.

I guess 1 minutes in previous version 0.18.x, you are using parallel.

Comparing parallel vs disableParallel() is expected got many times slower.

I will ๐Ÿ‘

I see. Weโ€™ll need to find out, which commit broke this for you, to avoid shooting in the dark.

Could you try few commits before 0.19.0 release and see, how it affects the speed? https://github.com/rectorphp/rector/commits/main/

image

{
    "require": {
         "rector/rector": "dev-main#dbbdf42842490a97fd9c35e4e899192e1b06f10b"
    }
}
composer update

FYI, i have 681 files and increased timeout by 60 seconds every time it died. Itโ€™s working now with 360 seconds. Before, it took about 30 seconds. I looked the release yesterday but donโ€™t find anything yet.

@vinceAmstoutz At first glance itโ€™s suspicious that your process of 800 files times out so soon. To compare Rector codebase has 2100 files and it finishes in about 40 seconds as whole.

If you can provide costly combination of rules and paths, we could look at the performance. We use this approach to find the files and rules - https://tomasvotruba.com/blog/2021/02/01/effective-debug-tricks-narrow-scoping/

I apply some performance improvement on latest dev-main, could you try update to "rector/rector": "dev-main" ?

If still happen, we may need a github repo to reproduce the issue, try use $rectorConfig->disableParallel() to verify the issue on specific rule or in parallel process.