snowball: KeyError: "Stemming algorithm 'porter' not found" on version 3.0.0 of the Python snowballstemmer package

The following code doesn’t work after upgrading to 3.0.0:

import snowballstemmer
snowballstemmer.stemmer('porter')

with the following error:

  File "/path/to/venv/lib/python3.13/site-packages/snowballstemmer/__init__.py", line 27, in stemmer
    raise KeyError("Stemming algorithm '%s' not found" % lang)
KeyError: "Stemming algorithm 'porter' not found"

It worked fine with v2.2.0. Tested on Python 3.13 and Python 3.11.

I don’t use snowballstemmer directly, but it’s used by Sphinx’s search functionality: https://github.com/sphinx-doc/sphinx/blob/c4929d026c8d22ba229b39cfc2250a9eb1476282/sphinx/search/en.py#L215

Not sure if this is a compatibility issue between Sphinx’s usage of the library or a genuine bug in the v3.0.0 release.

Thanks!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 43
  • Comments: 21 (9 by maintainers)

Commits related to this issue

Most upvoted comments

I am sorry for the breakage. I yanked version 3.0.0 for now, and will investigate and upload new version.

@laymonage: We didn’t inspect the codebase yet, but your evaluation sounds very likely. [^1]

Dear maintainers: Please yank this release from PyPI to resolve regressions on many downstream pipelines. 🙏

[^1]: We observed similar hiccups on corresponding transitions on a few other projects, where we haven’t been careful enough, and because regular CI workflows usually don’t validate against packaging flaws. 🍀

This is now fixed in version 3.0.0.1 on PyPI.

PR #231 contains a fix for the build system to prevent this from happening in the future.

Reminder to everyone, pinning is not needed now that it’s yanked. Also, you don’t need to pin, you could have just used snowballstemmer!=3.0.0, which protects against the broken version, but won’t come back to bite you later when something starts requiring >=3.

pydocstyle which uses snowballstemmer has similar issues. Those can also be reproduced by:

import snowballstemmer
stemmer = snowballstemmer.stemmer('english')

which raises the KeyError “Stemming algorithm ‘english’ not found”

I see this when using Flake8’s pydocstyle plugin:

File "~/repos/venvs/project/lib/python3.11/site-packages/flake8_docstrings.py", line 12, in <module>
    import pydocstyle as pep257
  File "~/repos/venvs/project/lib/python3.11/site-packages/pydocstyle/__init__.py", line 4, in <module>
    from .checker import ConventionChecker, check
  File "~/repos/venvs/project/lib/python3.11/site-packages/pydocstyle/checker.py", line 34, in <module>
    from .wordlists import IMPERATIVE_BLACKLIST, IMPERATIVE_VERBS, stem
  File "~/repos/venvs/project/lib/python3.11/site-packages/pydocstyle/wordlists.py", line 17, in <module>
    stem = snowballstemmer.stemmer('english').stemWord
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/repos/venvs/project/lib/python3.11/site-packages/snowballstemmer/__init__.py", line 27, in stemmer
    raise KeyError("Stemming algorithm '%s' not found" % lang)
KeyError: "Stemming algorithm 'english' not found"

And then there’s this:

$ python
Python 3.11.12 (main, Apr  9 2025, 08:55:54) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import snowballstemmer
>>> snowballstemmer.algorithms()
[]

Hi. That Sphinx/RTD build is also tripping on this.

We think it could be advisable to just yank the release from PyPI if it’s causing significant downstream issues on the many Sphinx installations/builds out there. Otherwise, would Sphinx be able to cope in any other way?

cc @ojwb – we’ve had a report of this in Sphinx, please let us know if we need to change our use of the API. I don’t see anything obvious in NEWS about this, though.

A

Probably better would be to fully script the update process so it starts from a clean checkout, but still run it locally

The fixes were in the creation process, there aren’t any code changes. So I’m not sure if there’s a need for a 3.0.1, 3.0.0.1 is fine unless there are other issues. The only problem was running make dist_libstemmer_python on a fresh checkout produced broken wheels, there was a required make step before it. I happened to run the required pre-step because I was just trying things, that’s why it worked for me. That’s protected now so future releases are safer.

I would recommend trying to make a proper build system that works from scratch without any pre-processing, of course, like scikit-build-core + cmake or hatchling and it’s build scripts. But that’s a bigger task. 😉

'm hesitant to do that because it would mean a compromise in the CI infrastructure could result in us unknowingly distributing releases with malicious code added

FYI, with trusted publishing, there are security benefits like SigStore signing that occur when publishing from CI. You can also protect the release environment and require a certain number of maintainers to sign off on it before it releases, for example. See https://learn.scientific-python.org/development/guides/gha-pure/ for an example of setup. Many of the PyPA’s packages use it, for example. It’s a different set of security issues, but not having to keep an all-powerful upload token and having a digital trail to your CI are some of the benefits.

Thanks! 3.0.0.1 worked for me 👍🏼

Hi again. @mitya57: Thank you for yanking the release, and thanks for the excellent hint about using snowballstemmer!=3.0.0 and for quickly finding the root cause, @henryiii.