pylibmc: "error 37 from memcached_set: SUCCESS" when setting value > 1 MB

With libmemcached 1.0.18 (OS X, with homebrew patches), I get an unexpected error value when attempting to set a value greater than 1 MB:

>>> import pylibmc 
>>> mc = pylibmc.Client(["127.0.0.1"])
>>> mc['foo'] = 'a' * 1024 * 1024
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pylibmc/src/pylibmc/client.py", line 163, in __setitem__
    if not self.set(key, value):
pylibmc.Error: error 37 from memcached_set: SUCCESS

I’m expecting an error, because of the 1 MB limit, but I was hoping for a more specific error, and maybe one that includes the length of the value after serialization and compression (see https://github.com/django-pylibmc/django-pylibmc/pull/29).

This sounds like a similar issue as #180, but without the binary protocol and behaviors used in that test case.

About this issue

Original URL
State: closed
Created 9 years ago
Comments: 20 (16 by maintainers)

Commits related to this issue

Add TooBig exception Refs #184 on Github — committed to lericson/pylibmc by lericson 8 years ago

Most upvoted comments

It’s the famous 1MB upper limit in memcached itself. I had to dig into this a few years ago, when the project was hosted on code.google.com and there was a FAQ with the explanation of why. I seem to remember it boiling down to “if you need to cache something that big, you are doing it wrong”, that TCP overhead would start to get ridiculous. It looks like you can increase it up to 128MB on the command line, if you have that server access, with something like memcached -I 128m. There may be a way to request the setting value over the API, if we wanted to do something smarter, but we’d have to guess at the overhead of storage structures, so it might not be worth it.

jwhitlock on Apr 11, 2016