ruff: Ruff should consider adding rules first as a warning instead of a block
First of all, thank you so much for working so hard to create and maintain Ruff, this is an amazing project and it has made our CI so much faster than using flak8 and other tools. This issue is not a bug, but a behavior that Ruff should consider before adding new rules/checks.
In this latest version, Ruff added the PLR1714
rule and broke our CI because the project I work on has code that this rule complains about - https://github.com/aws-powertools/powertools-lambda-python/pull/2822. In my opinion, this can be considered a breaking change as it has changed the behavior of how Ruff checks our code and evaluates rules.
That said, I would suggest that Ruff initially add a new rule/check as a warning for at least 2 or 3 releases before considering it a block, so we have time to fix the problems before breaking the CI.
Please let me know if I’m missing anything, but you should take this into account to improve the developer experience for Ruff customers.
Thanks
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 2
- Comments: 16 (9 by maintainers)
We’ve had a few discussions around this internally and have some initial ideas on what a more useful versioning scheme could look like. Our tentative plan is to queue this up as a project for next month – so we’ll draft a proposal internally, circulate it for public feedback and discussion, then put a transition plan in place.
I agree that ruff’s versioning scheme does not convey a lot of information. Even with ZeroVer one could distinguish between feature (potentially breaking) and bugfix releases. Is there a reason for only bumping patch versions?
However, I think it’s important to be precise and in my opinion adding a rule is not a breaking change. A breaking change would mean dropping or changing the meaning of CLI or configuration options. A new rule might prevent a pipeline from passing, but that’s due to a changed return code of the ruff invocation and not due to a break in ruff’s API. That’s also how Pylint is versioned.
I think this is an inherent “problem” of linters. What if a rule has bugs and does not identify problematic code? The bug gets fixed and suddenly your pipeline breaks because previously unidentified code smells are found. In my opinion, it’s on the user to at least explicitly control the applied rules or (even better) to pin a version and explicitly update it. This also means that you have control over when your pipeline breaks and not your linter’s release schedule.
@leandrodamascena - Heads up: we now have a proposal up for our versioning scheme available in https://github.com/astral-sh/ruff/discussions/6998
Thank you for bringing it up, it’s an important thing that we keep meaning to spend time on 😅
Yes, I’ll add it to my queue to post some thoughts here before the end of the week.
For what it is worth, Rome recommends doing this on their Getting Started and Versioning pages.
Thanks for raising this problem, and I’m sorry that we broke your CI.
Here are some thoughts of mine on this: