watchman: pywatchman raises SystemError on Python 3.10

At the risk of sounding like “that person” who suggests an alternative library that they maintain:

Those frustrated by watchman and watchdog might consider watchfiles.

A few advantages of watchfiles:

• It’s underlying file watching uses the heavily tested notify crate, when inotify / FSEvents / ReadDirectoryChangesW / kqueue are not available notify falls back to polling
• changes are denounced so changing branch results in one reload
• watchfiles is actively maintained and supports python >3.7, tests are already run with python 3.11
• it’s the default reloading library for uvicorn and there’s django-watchfiles for django users

Hey, sorry to be that developer, but can we have a release? pywatchman is broken on Python 3.10 until this is out.

@pauloxnet Apologies for the delay and I will try to get a version out soon.

I have published a pre-release 1.4.2.dev1 of pywatchman to TestPyPi. You can find it at https://test.pypi.org/project/pywatchman/1.4.2.dev1/. I am worried dumping four years of change to PyPi directly might break something somewhere so I’d like to take it slow. I also would like to find sometime to fix our CI and have the package automatically published some time in the future.

That being said, I have built some wheels for Python 3.6 to 3.10 targeting manylinux2014 and macos-arm64 there in TestPyPi (I built them by hand, not CI, thus the lack of Windows and macos-amd64 😦 ). Please use the following command to try it out. If everything works fine, I will make a pre-release to the mainline PyPi later this week for more testing.

pip install -i https://test.pypi.org/simple/ pywatchman==1.4.2.dev1

Thanks

---

Also, other people in this thread please give it a try as well. Thank you 😃

For anyone else that is waiting on the 1.4.2 release specifically for Django, you can “somewhat easily fix this” locally. I can’t imagine a case of doing this on a CI Server or something similar.

Locally you just edit the file .../site-packages/pywatchman/__init__.py and change line 1071.

Specifically the following

    def capabilityCheck(self, optional=None, required=None):
        """ Perform a server capability check """
        res = self.query('list-capabilities', {.  # <-- This says 'version' in the original 
            'optional': optional or [],
            'required': required or []
        })

Basically looks like watchman (the service) used to send capabilities when queried for version and it no longer does. Instead of you query for capabilities with list-capabilities the version is also sent. Subsequently there is some funny dictionary access that causes issues. This is why with Django 3.x and Python 3.9 and above WatchmanReloader doesn’t launch automatically.

I spent a bit of time following the rabbit down the hole to chase this issue… Hopefully you won’t have to… FWIW, this is a fairly trivial and harmless change IMO.

Small update I had to start a new service with Python 3.10 and this method almost works. It required further changes, which I was “curious” about and followed the rabbit. Sometimes on Mac it seems like the function _resolvesockname doesn’t quite work right. The cleanest thing you can do is just export the things it is looking for. Please remember these need to be exported into the process that is running your runserver (meaning if you are running it in IntelliJ or similar – you need to make sure these are available in that context)…

ps aux | grep watchman

This should give you the sock file something like this…

vnagendra         1632   0.0  0.2 37021908 136420   ??  S<   11Oct22   6:16.67 /usr/local/bin/watchman --foreground --logfile=/usr/local/var/run/watchman/vnagendra-state/log --log-level=1 --sockname=/usr/local/var/run/watchman/vnagendra-state/sock --statefile=/usr/local/var/run/watchman/vnagendra-state/state --pidfile=/usr/local/var/run/watchman/vnagendra-state/pid

Using that export the following

WATCHMAN_SOCK=/usr/local/var/run/watchman/vnagendra-state/sock
WATCHMAN_ENCODING=json

Once you resolve the socket problem, you’ll run into bser (their binary serialization method) having “issues” with python 3.10 – I didn’t quite troubleshoot that problem as much as just work around it.

I’m a user, not a maintainer. @fanzeyi is the only person who might make a release, it seems, but presumably they’re currently busy. I’m sure they’ll get to it when they can, but shipping a new release (to a certain extent) puts you on the hook for breakage. Right now they may not have the time to risk that.

@jwatt I add my opinion here but it is not a direct answer to you, indeed I thank you for the hint.

The last python release of watchman was in 2017, I don’t think a company like Facebook hasn’t had the time or other people to make a release in these, rather I think we have no interest in maintaining this python package.

The people who are commenting in this thread are writing and releasing free software everyday for the community and no one is making unreasonable requests.

The biggest problem, however, I do not think is a breakage, but the abandonment of the package by the users, after that of the developers themselves.

Personally I have long since removed the support of watchman in all my projects waiting for this release which by now I know it will not come or it will come too late.

The only regret is that Django supports out of the box watchman, I hope there will soon be Django support for a valid and maintained alternative such as watchfiles.

I don’t want to be that guy but, any updates on the release? @fanzeyi 👋🏻

@fanzeyi I have successfully tested pywatchman 1.4.2.dev1 in these environments:

• Ubuntu 21.10 and Debian 11
• watchman 4.9.0 (system package installed via apt)
• Python 3.9.7 and Python 3.10.0
• Django 3.2.7 and Django 3.2.10

I hope others will test and report here. Thanks.

FYI pywatchman 2.0.0 has been released and the original issue no longer reproduces on Python 3.11

$ python3.11 -m venv venv
$ . venv/bin/activate
$ pip install pywatchman==2.0.0
$ python3 -c "import pywatchman; print(pywatchman.client().capabilityCheck())"
# {'version': '2024.01.22.00', 'capabilities': {}}

@fanzeyi Could we please have a release? TestPyPI and similar sites are simply not an option if you have code that depends on this - you can’t just pull in stuff from TestPyPI in a requirements.txt/setup.cfg for example.

v1.4.2 also works for me on Python 3.10! I checked with Django and a standalone test script.

The same test passes on 3.12 and 3.13.0a4 as well.

AFAIK werkzeug[watchdog] isn’t as nice though since it doesn’t detect e.g. git operations and perform just a single reload instead of potentially many reloads for each file changed.

BTW, I pinged the developer on twitter and got some feedback on what’s blocking an updated release: https://twitter.com/fanzeyi/status/1537369006713217025

v1.4.2 works fine on my end, please release it ASAP.

@fanzeyi is more involved these days. I’ll sync up with them to ensure that they have the ability to update pypi

I’ll have a look at it as soon as possible. in the meantime i shared your request to test the dev version on twitter. I hope it helps this phase. https://twitter.com/pauloxnet/status/1475840504180924426?t=PNfe7Q3rGIN3S2ViChCAmA&s=19

I’m not familiar enough with watchman to even know where I can find pywatchman 1.4.1, but wasn’t this issue fixed in the code a year ago in https://github.com/facebook/watchman/commit/6813a948fee72a15acf4120262d37c9c8dc3f16b?