zulip: error 47 from memcached_set:

A recently restored server is sending this error to email but the server looks like it is functioning:

Logger django.pylibmc, from module django_pylibmc.memcached line 146:
Error generated by Anonymous user (not logged in) on zu2 deployment

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/e615b51dd743e2107a783e9c7780bdef53e26db3/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/sessions/backends/base.py", line 202, in _get_session
    return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/e615b51dd743e2107a783e9c7780bdef53e26db3/zulip-py3-venv/lib/python3.6/site-packages/django_pylibmc/memcached.py", line 130, in get
    return super(PyLibMCCache, self).get(key, default, version)
  File "/srv/zulip-venv-cache/e615b51dd743e2107a783e9c7780bdef53e26db3/zulip-py3-venv/lib/python3.6/site-packages/django/core/cache/backends/memcached.py", line 79, in get
    val = self._cache.get(key)
pylibmc.ServerDown: error 47 from memcached_get(:1:django.contrib.sessions.cache): (0x560466139190) SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY, 127.0.0.1:11211,  host: 127.0.0.1:11211 -> libmemcached/connect.cc:811

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/e615b51dd743e2107a783e9c7780bdef53e26db3/zulip-py3-venv/lib/python3.6/site-packages/django_pylibmc/memcached.py", line 140, in set
    **COMPRESS_KWARGS)
pylibmc.ServerDown: error 47 from memcached_set: (0x560466139190) SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY,  host: 127.0.0.1:11211 -> libmemcached/connect.cc:720


Deployed code:
- git: 2.1.0-289-gb23a5431cd
- ZULIP_VERSION: 2.1.0-289-gb23a5431cd


Request info: none

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 47 (18 by maintainers)

Commits related to this issue

puppet: Fix regeneration of memcached-sasldb2 on password changes. Puppet doesn’t re-run an exec blocks that’s declared as creating an existing file, even if it’s notified. Remove the creates declar... — committed to andersk/zulip by andersk 4 years ago
puppet: Fix regeneration of memcached-sasldb2 on password changes. Puppet doesn’t re-run an exec blocks that’s declared as creating an existing file, even if it’s notified. Remove the creates declar... — committed to zulip/zulip by andersk 4 years ago
puppet: Fix regeneration of memcached-sasldb2 on password changes. Puppet doesn’t re-run an exec blocks that’s declared as creating an existing file, even if it’s notified. Remove the creates declar... — committed to zulip/zulip by andersk 4 years ago
puppet: Fix regeneration of memcached-sasldb2 on password changes. Puppet doesn’t re-run an exec blocks that’s declared as creating an existing file, even if it’s notified. Remove the creates declar... — committed to stableapple/zulip by andersk 4 years ago

Most upvoted comments

We just merged a fix for the memcached issue here; thanks for the reports! It’ll be in the 2.1.3 release and I’m adding it to the 2.1.x branch, so you can run upgrade-zulip-from-git 2.1.x to verify it.

timabbott on Feb 19, 2020

@timabbott Actually I think the error now is a new one. It says:

error 47 from memcached_set: (0x55ce45879f80) SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY, host: 127.0.0.1:11211 -> libmemcached/connect.cc:720

stone212 on Jan 28, 2020

The exception is caused by the memcached process having been restarted. The most common cause is an apt upgrade restarting memcached without your having restarting the Zulip server afterwards; many details are here: https://zulip.readthedocs.io/en/latest/production/troubleshooting.html#disabling-unattended-upgrades

timabbott on Jan 24, 2020