pants: bug with loading dependencies when running mypy

Pants version 2.16, but also was happening on 2.15

OS Mac

I have been trying to hunt down a bug in pants for a few weeks and finally have a reproduction, but don’t know what do do with it.

From time to time, a mypy run will fail due to missing dependencies. Dependencies which are definitely configured and findable. Often a re-run will succeed, which has made this very hard to reproduce.

I now found a case where running a single file will fail, but running with :: will succeed. I have both sandboxes and when I edit __files.txt in the latter it passes. However I don’t know what to look for in these sandboxes to figure out what’s going on.

Failure from the “bad” sandbox:

.cache/pex_root/venvs/e1862d6434ffd1c4230c239d7d17d05591664247/4e8b276cfa3c3338d49f8656ee098b9ee6162291/lib/python3.8/site-packages/botocore-stubs/compat.pyi:20:1: error: Library stubs not installed for "six.moves.http_client"  [import]
    from six.moves import http_client
    ^
.cache/pex_root/venvs/e1862d6434ffd1c4230c239d7d17d05591664247/4e8b276cfa3c3338d49f8656ee098b9ee6162291/lib/python3.8/site-packages/botocore-stubs/compat.pyi:20:1: note: Hint: "python3 -m pip install types-six"
.cache/pex_root/venvs/e1862d6434ffd1c4230c239d7d17d05591664247/4e8b276cfa3c3338d49f8656ee098b9ee6162291/lib/python3.8/site-packages/botocore-stubs/compat.pyi:20:1: note: (or run "mypy --install-types" to install all missing stub packages)
.cache/pex_root/venvs/e1862d6434ffd1c4230c239d7d17d05591664247/4e8b276cfa3c3338d49f8656ee098b9ee6162291/lib/python3.8/site-packages/botocore-stubs/compat.pyi:20:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
.cache/pex_root/venvs/e1862d6434ffd1c4230c239d7d17d05591664247/4e8b276cfa3c3338d49f8656ee098b9ee6162291/lib/python3.8/site-packages/botocore-stubs/compat.pyi:20:1: error: Library stubs not installed for "six.moves"  [import]
    from six.moves import http_client
    ^
src/django_apps/counseling/google_calendar.py:74:14: error: Module has no attribute "parser"  [attr-defined]
                (dateutil.parser.parse(e['start']), dateutil.parser.parse(e['end']))
                 ^~~~~~~~~~~~~~~
Found 3 errors in 2 files (checked 1 source file)

Success from the good sandbox, edited to run only the file in question:

Success: no issues found in 1 source file

A fun complication is that the mypy cache appears to get poisoned. If I have a failed run on the “bad” sandbox, the good one will then fail. If I reset the cache it will work.

About this issue

Original URL
State: open
Created a year ago
Reactions: 2
Comments: 15 (9 by maintainers)

Most upvoted comments

FYI,

# pants.toml
[mypy]
args = ["--no-incremental"]

Should fix it until this bug get fixed in mypy.

isra17 on Mar 20, 2024

I’m running into some flakiness with mypy too. Blowing away the cache is unnecessary, merely re-running pants check is sufficient. I am running into this both in pre-commit as well as just running pants check from the command line.

I am getting this error with other modules than mentioned here:

.cache/pex_root/venvs/d17349559f692efab258d5f40c73f81d7382b44c/2540ac6a914a2a78bfd89ca0676ffdc7f0dfe44f/lib/python3.11/site-packages/pandas-stubs/io/formats/style_render.pyi:11: error: Cannot find implementation or library stub for module named "jinja2"  [import-not-found]
.cache/pex_root/venvs/d17349559f692efab258d5f40c73f81d7382b44c/2540ac6a914a2a78bfd89ca0676ffdc7f0dfe44f/lib/python3.11/site-packages/opentelemetry/instrumentation/requests/__init__.py:57: error: Library stubs not installed for "requests.models"  [import-untyped]

I have a separate lockfile for mypy due to some issue with the version of mypy that is bundled with pants, no idea if related.

The error seems relatively new, I don’t recall seeing anything like this pre 2.19.0

kuza55 on Mar 19, 2024

This may be related with the “–changed” subsystem.

achimnol on May 26, 2023