jsonpickle: Test failures with Python 3.11

Hello. We are trying to rebuild jsonpickle in Fedora with Python 3.11.0b3. Currently, there are various test failures in our build environment, but I can reproduce many with plan tox. To avoid dealing with compilation issues of scipy and pandas, I’ve reduced the tests like this:

diff --git a/setup.cfg b/setup.cfg
index f068a00..7712763 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -57,14 +57,7 @@ testing =
     pytest-cov
 
     # local
-    ecdsa
-    feedparser
-    gmpy2
-    numpy
-    pandas
-    pymongo
-    scikit-learn
-    sqlalchemy
+    pytz

And run tox -e py311 -- -k 'not (test_thing_with_module or test_thing_with_submodule)' tests 2>&1 | tee tox_output.

(test_thing_with_module and test_thing_with_submodule hang for me when reproducing like this, so I’ve deselected them)

This is the output:

WARNING: The wheel package is not available.
py311 develop-inst-noop: .../jsonpickle
py311 installed: attrs==21.4.0,black==22.3.0,click==8.1.3,coverage==6.4.1,docutils==0.18.1,flake8==4.0.1,importlib-metadata==4.11.4,iniconfig==1.1.1,-e git+ssh://git@github.com/jsonpickle/jsonpickle.git@c2fdc01887d6173fdd46e0d322832b906ee80e22#egg=jsonpickle,mccabe==0.6.1,mypy-extensions==0.4.3,packaging==21.3,pathspec==0.9.0,pep517==0.12.0,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pycodestyle==2.8.0,pyflakes==2.4.0,pyparsing==3.0.9,pytest==7.1.2,pytest-black==0.3.12,pytest-black-multipy==1.0.1,pytest-checkdocs==2.7.1,pytest-cov==3.0.0,pytest-flake8==1.1.1,pytz==2022.1,toml==0.10.2,tomli==2.0.1,zipp==3.8.0
py311 run-test-pre: PYTHONHASHSEED='573023578'
py311 run-test: commands[0] | pytest -k 'not (test_thing_with_module or test_thing_with_submodule)' tests
============================= test session starts ==============================
platform linux -- Python 3.11.0b3, pytest-7.1.2, pluggy-1.0.0
cachedir: .tox/py311/.pytest_cache
rootdir: .../jsonpickle, configfile: pytest.ini
plugins: checkdocs-2.7.1, flake8-1.1.1, cov-3.0.0, black-0.3.12, black-multipy-1.0.1
collected 296 items / 3 deselected / 6 skipped / 293 selected

tests/backend_test.py s.....ssssssssss                                   [  5%]
tests/benchmark.py s                                                     [  5%]
tests/bson_test.py sssssss                                               [  8%]
tests/collections_test.py sFFF.FF                                        [ 10%]
tests/datetime_test.py s................                                 [ 16%]
tests/document_test.py sF                                                [ 17%]
tests/ecdsa_test.py sE                                                   [ 17%]
tests/feedparser_test.py ss                                              [ 18%]
tests/handler_test.py s......                                            [ 20%]
tests/helper.py s                                                        [ 21%]
tests/jsonpickle_test.py s......FFFF........................F.F.......F. [ 37%]
................F..F.............................FF........F             [ 57%]
tests/numpy_test.py s                                                    [ 58%]
tests/object_test.py s...........F..FF........FF..FFF.....FFFFFF.....F.. [ 75%]
.F....XXX......F.........                                                [ 83%]
tests/pandas_test.py s                                                   [ 84%]
tests/sklearn_test.py s                                                  [ 84%]
tests/sqlalchemy_test.py ssss                                            [ 86%]
tests/stdlib_test.py s...                                                [ 87%]
tests/util_test.py s.................................                    [ 98%]
tests/wizard_test.py sF.                                                 [100%]

==================================== ERRORS ====================================
________________ ERROR at setup of EcdsaTestCase.test_roundtrip ________________
...
=================================== FAILURES ===================================
______________________________ test_dict_no_cycle ______________________________

    def test_dict_no_cycle():
        g = Group('group')
        c1 = C(42)
        g.elements.append(c1)
        c2 = C(67)
        g.elements.append(c2)
        c1.add(c2, 'a')  # points to c2, which does not point to anything
    
        assert c2 in c1.plain
        assert c2 in c1.plain_ordered
        assert c2 in c1.plain_default
    
        gu = pickle_and_unpickle(g)
        c1u = gu.elements[0]
        c2u = gu.elements[1]
    
        # check existence of keys directly
        assert c2u in c1u.plain.keys()
        assert c2u in c1u.plain_ordered.keys()
>       assert c2u in c1u.plain_default.keys()
E       AttributeError: 'NoneType' object has no attribute 'keys'

tests/collections_test.py:139: AttributeError
_____________________________ test_dict_self_cycle _____________________________

    def test_dict_self_cycle():
        g = Group('group')
        c1 = C(42)
        g.elements.append(c1)
        c2 = C(67)
        g.elements.append(c2)
        c1.add(c1, 'a')  # cycle to itself
        c1.add(c2, 'b')  # c2 does not point to itself nor c1
    
        assert c1 in c1.plain
        assert c1 in c1.plain_ordered
        assert c1 in c1.plain_default
    
>       gu = pickle_and_unpickle(g)

tests/collections_test.py:171: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/collections_test.py:117: in pickle_and_unpickle
    return jsonpickle.decode(encoded, keys=True)
jsonpickle/unpickler.py:88: in decode
    return context.restore(data, reset=reset, classes=classes)
jsonpickle/unpickler.py:362: in restore
    value = self._restore(obj)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:652: in _restore_state
    state = self._restore(obj[tags.STATE])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:798: in _restore_dict
    data[k] = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:431: in _restore_list
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:431: in <listcomp>
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:663: in _restore_state
    instance = self._restore_from_dict(state, instance, ignorereserved=False)
jsonpickle/unpickler.py:612: in _restore_from_dict
    value = self._restore(v)
jsonpickle/unpickler.py:343: in _restore
    restore = self._restore_tags(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <jsonpickle.unpickler.Unpickler object at 0x7f269470e050>
obj = <[KeyError(C(42)) raised in repr()] OrderedDict object at 0x7f26946f8dc0>

    def _restore_tags(self, obj):
        try:
>           if not tags.RESERVED <= set(obj) and not type(obj) in (list, dict):
E           KeyError: C(42)

jsonpickle/unpickler.py:832: KeyError
____________________________ test_dict_mutual_cycle ____________________________

    def test_dict_mutual_cycle():
        g = Group('group')
        c1 = C(42)
        g.elements.append(c1)
        c2 = C(67)
        g.elements.append(c2)
    
        c1.add(c2, 'a')  # points to c2, which points to c1, forming cycle
        c2.add(c1, 'a')  # points to c1 in order to form cycle
    
        assert c2 in c1.plain
        assert c2 in c1.plain_ordered
        assert c2 in c1.plain_default
    
        assert c1 in c2.plain
        assert c1 in c2.plain_ordered
        assert c1 in c2.plain_default
    
        gu = pickle_and_unpickle(g)
        c1u = gu.elements[0]
        c2u = gu.elements[1]
    
        # check existence of keys directly
        # key c2u
        assert c2u in c1u.plain.keys()
        assert c2u in c1u.plain_ordered.keys()
>       assert c2u in c1u.plain_default.keys()
E       AttributeError: 'NoneType' object has no attribute 'keys'

tests/collections_test.py:252: AttributeError
_____________________________ test_set_self_cycle ______________________________

    def test_set_self_cycle():
        g = Group('group')
        d1 = D(42)
        g.elements.append(d1)
        d2 = D(67)
        g.elements.append(d2)
        d1.add(d1)  # cycle to itself
        d1.add(d2)  # d1 also points to d2, but d2 does not point to d1
    
        assert d1 in d1.plain
    
        gu = pickle_and_unpickle(g)
        d1u = gu.elements[0]
        d2u = gu.elements[1]
        assert d2u is not None
    
        # check element directly
>       assert d1u in d1u.plain
E       assert D(42) in {D(42), D(67)}
E        +  where {D(42), D(67)} = D(42).plain

tests/collections_test.py:335: AssertionError
____________________________ test_set_mutual_cycle _____________________________

    def test_set_mutual_cycle():
        g = Group('group')
        d1 = D(42)
        g.elements.append(d1)
        d2 = D(67)
        g.elements.append(d2)
        d1.add(d2)  # points to d2, which points to d1, forming cycle
        d2.add(d1)  # points to d1 in order to form cycle
    
        assert d2 in d1.plain
        assert d1 in d2.plain
    
        gu = pickle_and_unpickle(g)
        d1u = gu.elements[0]
        d2u = gu.elements[1]
    
        # check element directly
        # succeeds because d2u added to d1u after __setstate__
        assert d2u in d1u.plain
>       assert d1u in d2u.plain
E       assert D(42) in {D(42)}
E        +  where {D(42)} = D(67).plain

tests/collections_test.py:361: AssertionError
________________________ DocumentTestCase.test_cyclical ________________________

self = <document_test.DocumentTestCase testMethod=test_cyclical>

    def test_cyclical(self):
        """Test that we can pickle cyclical data structure
    
        This test is ensures that we can reference objects which
        first appear within a list (in other words, not a top-level
        object or attribute).  Later children will reference that
        object through its "_parent" field.
    
        This makes sure that we handle this case correctly.
    
        """
        document = Document('My Document')
        section1 = Section('Section 1')
        section2 = Section('Section 2')
        question1 = Question('Question 1')
        question2 = Question('Question 2')
        question3 = Question('Question 3')
        question4 = Question('Question 4')
    
        document.add_child(section1)
        document.add_child(section2)
        section1.add_child(question1)
        section1.add_child(question2)
        section2.add_child(question3)
        section2.add_child(question4)
    
        pickled = jsonpickle.encode(document)
        unpickled = jsonpickle.decode(pickled)
    
>       self.assertEqual(str(document), str(unpickled))

tests/document_test.py:90: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/document_test.py:31: in __str__
    ret_str += repr(c)
tests/document_test.py:57: in __repr__
    return self.__str__()
tests/document_test.py:53: in __str__
    ret_str += repr(c)
tests/document_test.py:43: in __repr__
    return self.__str__()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'list' object has no attribute '_name'") raised in repr()] Question object at 0x7f2694753cd0>

    def __str__(self):
>       return 'Question "%s", parent: "%s"\n' % (self._name, self._parent._name)
E       AttributeError: 'list' object has no attribute '_name'

tests/document_test.py:40: AttributeError
_________________________ PicklingTestCase.test_class __________________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_class>

    def test_class(self):
        inst = Thing('test name')
        inst.child = Thing('child name')
    
        flattened = self.pickler.flatten(inst)
>       self.assertEqual('test name', flattened['name'])
E       KeyError: 'name'

tests/jsonpickle_test.py:300: KeyError
____________________ PicklingTestCase.test_class_reference _____________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_class_reference>

    def test_class_reference(self):
        """This test ensures that users can store references to classes."""
        obj = Thing('object-with-class-reference')
    
        # reference the 'Thing' class (not an instance of the class)
        obj.classref = Thing
    
        flattened = self.pickler.flatten(obj)
>       self.assertEqual(flattened['classref'], {tags.TYPE: 'jsonpickle_test.Thing'})
E       KeyError: 'classref'

tests/jsonpickle_test.py:424: KeyError
_______________________ PicklingTestCase.test_classdict ________________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_classdict>

    def test_classdict(self):
        dict = {'k1': Thing('one'), 'k2': Thing('two'), 'k3': 3}
    
        flattened = self.pickler.flatten(dict)
>       self.assertEqual('one', flattened['k1']['name'])
E       KeyError: 'name'

tests/jsonpickle_test.py:329: KeyError
_______________________ PicklingTestCase.test_classlist ________________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_classlist>

    def test_classlist(self):
        array = [Thing('one'), Thing('two'), 'a string']
    
        flattened = self.pickler.flatten(array)
>       self.assertEqual('one', flattened[0]['name'])
E       KeyError: 'name'

tests/jsonpickle_test.py:314: KeyError
_____________________ PicklingTestCase.test_type_reference _____________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_type_reference>

    def test_type_reference(self):
        """This test ensures that users can store references to types."""
        obj = Thing('object-with-type-reference')
    
        # reference the built-in 'object' type
        obj.typeref = object
    
        flattened = self.pickler.flatten(obj)
>       self.assertEqual(flattened['typeref'], {tags.TYPE: 'builtins.object'})
E       KeyError: 'typeref'

tests/jsonpickle_test.py:411: KeyError
__________________ PicklingTestCase.test_unpickler_on_missing __________________

self = <jsonpickle_test.PicklingTestCase testMethod=test_unpickler_on_missing>

    def test_unpickler_on_missing(self):
        class SimpleClass(object):
            def __init__(self, i):
                self.i = i
    
        frozen = jsonpickle.encode(SimpleClass(4))
        del SimpleClass
    
        # https://docs.python.org/3/library/warnings.html#testing-warnings
    
        with warnings.catch_warnings(record=True) as w:
            warnings.simplefilter("always")
            jsonpickle.decode(frozen, on_missing='warn')
            self.assertTrue(issubclass(w[-1].category, UserWarning))
            self.assertTrue(
                "Unpickler._restore_object could not find" in str(w[-1].message)
            )
    
            jsonpickle.decode(frozen, on_missing=on_missing_callback)
            self.assertTrue(issubclass(w[-1].category, RuntimeWarning))
            self.assertTrue("The unpickler couldn't find" in str(w[-1].message))
    
>       self.assertTrue(
            jsonpickle.decode(frozen, on_missing='ignore')
            == {
                'py/object': 'jsonpickle_test.PicklingTestCase.test_unpickler_on_missing.<locals>.SimpleClass',
                'i': 4,
            }
        )
E       AssertionError: False is not true

tests/jsonpickle_test.py:522: AssertionError
____________________ JSONPickleTestCase.test_dict_subclass _____________________

self = <jsonpickle_test.JSONPickleTestCase testMethod=test_dict_subclass>

    def test_dict_subclass(self):
        obj = UserDict()
        obj.valid = True
        obj.s = 'string'
        obj.d = 'd_string'
        obj['d'] = {}
        obj['s'] = 'test'
        pickle = jsonpickle.encode(obj)
        actual = jsonpickle.decode(pickle)
    
        self.assertEqual(type(actual), UserDict)
        self.assertTrue('d' in actual)
        self.assertTrue('s' in actual)
>       self.assertTrue(hasattr(actual, 'd'))
E       AssertionError: False is not true

tests/jsonpickle_test.py:713: AssertionError
__________________ JSONPickleTestCase.test_reference_to_list ___________________

self = <jsonpickle_test.JSONPickleTestCase testMethod=test_reference_to_list>

    def test_reference_to_list(self):
        thing = Thing('parent')
        thing.a = [1]
        thing.b = thing.a
        thing.b.append(thing.a)
        thing.b.append([thing.a])
    
        encoded = jsonpickle.encode(thing)
>       decoded = jsonpickle.decode(encoded)

tests/jsonpickle_test.py:858: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
jsonpickle/unpickler.py:88: in decode
    return context.restore(data, reset=reset, classes=classes)
jsonpickle/unpickler.py:362: in restore
    value = self._restore(obj)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:663: in _restore_state
    instance = self._restore_from_dict(state, instance, ignorereserved=False)
jsonpickle/unpickler.py:612: in _restore_from_dict
    value = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:431: in _restore_list
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:431: in <listcomp>
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
E   RecursionError: maximum recursion depth exceeded in __instancecheck__
!!! Recursion detected (same locals & position)
____________________ JSONPickleTestCase.test_refs_recursive ____________________

self = <jsonpickle.unpickler.Unpickler object at 0x7f26943a5110>, obj = {}

    def _mkref(self, obj):
        obj_id = id(obj)
        try:
>           self._obj_to_idx[obj_id]
E           KeyError: 139803671999104

jsonpickle/unpickler.py:419: KeyError

During handling of the above exception, another exception occurred:

self = <jsonpickle_test.JSONPickleTestCase testMethod=test_refs_recursive>

    def test_refs_recursive(self):
        """Test that complicated recursive refs work"""
    
        a = Thing('a')
        a.self_list = [Thing('0'), Thing('1'), Thing('2')]
        a.first = a.self_list[0]
        a.stuff = {a.first: a.first}
        a.morestuff = {a.self_list[1]: a.stuff}
    
        pickle = jsonpickle.encode(a, keys=True)
>       b = jsonpickle.decode(pickle, keys=True)

tests/jsonpickle_test.py:773: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
jsonpickle/unpickler.py:88: in decode
    return context.restore(data, reset=reset, classes=classes)
jsonpickle/unpickler.py:362: in restore
    value = self._restore(obj)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:663: in _restore_state
    instance = self._restore_from_dict(state, instance, ignorereserved=False)
jsonpickle/unpickler.py:612: in _restore_from_dict
    value = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:798: in _restore_dict
    data[k] = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:782: in _restore_dict
    self._mkref(data)
jsonpickle/unpickler.py:425: in _mkref
    self._namedict[self._refname()] = obj
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <jsonpickle.unpickler.Unpickler object at 0x7f26943a5110>

    def _refname(self):
        """Calculates the name of the current location in the JSON stack.
    
        This is called as jsonpickle traverses the object structure to
        create references to previously-traversed objects.  This allows
        cyclical data structures such as doubly-linked lists.
        jsonpickle ensures that duplicate python references to the same
        object results in only a single JSON object definition and
        special reference tags to represent each reference.
    
        >>> u = Unpickler()
        >>> u._namestack = []
        >>> u._refname() == '/'
        True
        >>> u._namestack = ['a']
        >>> u._refname() == '/a'
        True
        >>> u._namestack = ['a', 'b']
        >>> u._refname() == '/a/b'
        True
    
        """
>       return '/' + '/'.join(self._namestack)
E       TypeError: sequence item 1: expected str instance, Thing found

jsonpickle/unpickler.py:414: TypeError
_______________ PicklingProtocol2TestCase.test_reduce_state_dict _______________

self = <jsonpickle_test.PicklingProtocol2TestCase testMethod=test_reduce_state_dict>

    def test_reduce_state_dict(self):
        """Test reduce with the optional state argument set,
        on an object with a __dict__, and no __setstate__"""
    
        instance = PickleProtocol2ReduceTupleState(5)
        encoded = jsonpickle.encode(instance)
        decoded = jsonpickle.decode(encoded)
>       self.assertEqual(decoded.argval, 'yam')
E       AssertionError: 5 != 'yam'

tests/jsonpickle_test.py:1372: AssertionError
_____________ PicklingProtocol2TestCase.test_reduce_state_no_dict ______________

self = <jsonpickle_test.PicklingProtocol2TestCase testMethod=test_reduce_state_no_dict>

    def test_reduce_state_no_dict(self):
        """Test reduce with the optional state argument set,
        on an object with no __dict__, and no __setstate__"""
    
        instance = PickleProtocol2ReduceTupleStateSlots(5)
        encoded = jsonpickle.encode(instance)
        decoded = jsonpickle.decode(encoded)
>       self.assertEqual(decoded.argval, 'yam')
E       AssertionError: 5 != 'yam'

tests/jsonpickle_test.py:1361: AssertionError
_______________________ test_repeat_objects_are_expanded _______________________

    def test_repeat_objects_are_expanded():
        """Ensure that all objects are present in the json output"""
        # When references are disabled we should create expanded copies
        # of any object that appears more than once in the object stream.
        alice = Thing('alice')
        bob = Thing('bob')
        alice.child = bob
    
        car = Thing('car')
        car.driver = alice
        car.owner = alice
        car.passengers = [alice, bob]
    
        pickler = jsonpickle.Pickler(make_refs=False)
        flattened = pickler.flatten(car)
    
>       assert flattened['name'] == 'car'
E       KeyError: 'name'

tests/jsonpickle_test.py:1659: KeyError
______________ AdvancedObjectsTestCase.test_base_object_roundrip _______________

self = <object_test.AdvancedObjectsTestCase testMethod=test_base_object_roundrip>

    def test_base_object_roundrip(self):
        roundtrip = self.unpickler.restore(self.pickler.flatten(object()))
>       self.assertEqual(type(roundtrip), object)
E       AssertionError: <class 'NoneType'> != <class 'object'>

tests/object_test.py:494: AssertionError
__________________ AdvancedObjectsTestCase.test_dictsubclass ___________________

self = <object_test.AdvancedObjectsTestCase testMethod=test_dictsubclass>

    def test_dictsubclass(self):
        obj = DictSubclass()
        obj['key1'] = 1
    
        expect = {
            tags.OBJECT: 'object_test.DictSubclass',
            'key1': 1,
            '__dict__': {},
        }
        flattened = self.pickler.flatten(obj)
>       self.assertEqual(expect, flattened)
E       AssertionError: {'py/object': 'object_test.DictSubclass', 'key1': 1, '__dict__': {}} != {'py/object': 'object_test.DictSubclass', 'py/state': None}
E       - {'__dict__': {}, 'key1': 1, 'py/object': 'object_test.DictSubclass'}
E       + {'py/object': 'object_test.DictSubclass', 'py/state': None}

tests/object_test.py:367: AssertionError
___________ AdvancedObjectsTestCase.test_dictsubclass_notunpickable ____________

self = <object_test.AdvancedObjectsTestCase testMethod=test_dictsubclass_notunpickable>

    def test_dictsubclass_notunpickable(self):
        self.pickler.unpicklable = False
    
        obj = DictSubclass()
        obj['key1'] = 1
    
        flattened = self.pickler.flatten(obj)
>       self.assertEqual(1, flattened['key1'])
E       TypeError: 'NoneType' object is not subscriptable

tests/object_test.py:381: TypeError
__________________ AdvancedObjectsTestCase.test_list_subclass __________________

self = <object_test.AdvancedObjectsTestCase testMethod=test_list_subclass>

    def test_list_subclass(self):
        obj = ListSubclass()
        obj.extend([1, 2, 3])
        flattened = self.pickler.flatten(obj)
        self.assertTrue(tags.OBJECT in flattened)
>       self.assertTrue(tags.SEQ in flattened)
E       AssertionError: False is not true

tests/object_test.py:293: AssertionError
_____________ AdvancedObjectsTestCase.test_list_subclass_with_data _____________

self = <object_test.AdvancedObjectsTestCase testMethod=test_list_subclass_with_data>

    def test_list_subclass_with_data(self):
        obj = ListSubclass()
        obj.extend([1, 2, 3])
        data = SetSubclass([1, 2, 3])
        obj.data = data
        flattened = self.pickler.flatten(obj)
        restored = self.unpickler.restore(flattened)
>       self.assertEqual(restored, obj)
E       AssertionError: [] != [1, 2, 3]

tests/object_test.py:315: AssertionError
__________________ AdvancedObjectsTestCase.test_oldstyleclass __________________

self = <object_test.AdvancedObjectsTestCase testMethod=test_oldstyleclass>

    def test_oldstyleclass(self):
        obj = OldStyleClass()
        obj.value = 1234
    
        flattened = self.pickler.flatten(obj)
>       self.assertEqual(1234, flattened['value'])
E       KeyError: 'value'

tests/object_test.py:352: KeyError
__________________ AdvancedObjectsTestCase.test_set_subclass ___________________

self = <object_test.AdvancedObjectsTestCase testMethod=test_set_subclass>

    def test_set_subclass(self):
        obj = SetSubclass([1, 2, 3])
        flattened = self.pickler.flatten(obj)
        self.assertTrue(tags.OBJECT in flattened)
>       self.assertTrue(tags.SEQ in flattened)
E       AssertionError: False is not true

tests/object_test.py:323: AssertionError
_____________ AdvancedObjectsTestCase.test_set_subclass_with_data ______________

self = <object_test.AdvancedObjectsTestCase testMethod=test_set_subclass_with_data>

    def test_set_subclass_with_data(self):
        obj = SetSubclass([1, 2, 3])
        data = ListSubclass()
        data.extend([1, 2, 3])
        obj.data = data
        flattened = self.pickler.flatten(obj)
        restored = self.unpickler.restore(flattened)
>       self.assertEqual(restored.data.__class__, ListSubclass)
E       AssertionError: <class 'NoneType'> != <class 'object_test.ListSubclass'>

tests/object_test.py:338: AssertionError
__________________________ test_defaultdict_roundtrip __________________________

    def test_defaultdict_roundtrip():
        """Make sure we can handle collections.defaultdict(list)"""
        # setup
        defaultdict = collections.defaultdict
        defdict = defaultdict(list)
        defdict['a'] = 1
        defdict['b'].append(2)
        defdict['c'] = defaultdict(dict)
        # jsonpickle work your magic
        encoded = jsonpickle.encode(defdict)
        newdefdict = jsonpickle.decode(encoded)
        # jsonpickle never fails
>       assert newdefdict['a'] == 1
E       TypeError: 'NoneType' object is not subscriptable

tests/object_test.py:630: TypeError
___________________ test_defaultdict_roundtrip_simple_lambda ___________________

    def test_defaultdict_roundtrip_simple_lambda():
        """Make sure we can handle defaultdict(lambda: defaultdict(int))"""
        # setup a sparse collections.defaultdict with simple lambdas
        defaultdict = collections.defaultdict
        defdict = defaultdict(lambda: defaultdict(int))
        defdict[0] = 'zero'
        defdict[1] = defaultdict(lambda: defaultdict(dict))
        defdict[1][0] = 'zero'
        # roundtrip
        encoded = jsonpickle.encode(defdict, keys=True)
        newdefdict = jsonpickle.decode(encoded, keys=True)
>       assert newdefdict[0] == 'zero'
E       TypeError: 'NoneType' object is not subscriptable

tests/object_test.py:648: TypeError
__________________ test_defaultdict_roundtrip_simple_lambda2 ___________________

    def test_defaultdict_roundtrip_simple_lambda2():
        """Serialize a defaultdict that contains a lambda"""
        defaultdict = collections.defaultdict
        payload = {'a': defaultdict(lambda: 0)}
        defdict = defaultdict(lambda: 0, payload)
        # roundtrip
        encoded = jsonpickle.encode(defdict, keys=True)
        decoded = jsonpickle.decode(encoded, keys=True)
>       assert type(decoded) == defaultdict
E       AssertionError: assert <class 'NoneType'> == <class 'collections.defaultdict'>
E        +  where <class 'NoneType'> = type(None)

tests/object_test.py:666: AssertionError
_____________ test_defaultdict_and_things_roundtrip_simple_lambda ______________

    def test_defaultdict_and_things_roundtrip_simple_lambda():
        """Serialize a default dict that contains a lambda and objects"""
        thing = Thing('a')
        defaultdict = collections.defaultdict
        defdict = defaultdict(lambda: 0)
        obj = [defdict, thing, thing]
        # roundtrip
        encoded = jsonpickle.encode(obj, keys=True)
        decoded = jsonpickle.decode(encoded, keys=True)
>       assert decoded[0].default_factory() == 0
E       AttributeError: 'NoneType' object has no attribute 'default_factory'

tests/object_test.py:679: AttributeError
____________ test_defaultdict_subclass_with_self_as_default_factory ____________

    def test_defaultdict_subclass_with_self_as_default_factory():
        """Serialize a defaultdict subclass with self as its default factory"""
        cls = ThingWithSelfAsDefaultFactory
        tree = cls()
>       newtree = _test_defaultdict_tree(tree, cls)

tests/object_test.py:711: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

tree = ThingWithSelfAsDefaultFactory(ThingWithSelfAsDefaultFactory(..., {...}), {'A': ThingWithSelfAsDefaultFactory(ThingWithSelfAsDefaultFactory(..., {...}), {'B': 1, 'C': 2})})
cls = <class 'object_test.ThingWithSelfAsDefaultFactory'>

    def _test_defaultdict_tree(tree, cls):
        tree['A']['B'] = 1
        tree['A']['C'] = 2
        # roundtrip
        encoded = jsonpickle.encode(tree)
        newtree = jsonpickle.decode(encoded)
        # make sure we didn't lose anything
>       assert type(newtree) == cls
E       AssertionError: assert <class 'NoneType'> == <class 'object_test.ThingWithSelfAsDefaultFactory'>
E        +  where <class 'NoneType'> = type(None)

tests/object_test.py:690: AssertionError
___________ test_defaultdict_subclass_with_class_as_default_factory ____________

    def test_defaultdict_subclass_with_class_as_default_factory():
        """Serialize a defaultdict with a class as its default factory"""
        cls = ThingWithClassAsDefaultFactory
        tree = cls()
>       newtree = _test_defaultdict_tree(tree, cls)

tests/object_test.py:722: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

tree = ThingWithClassAsDefaultFactory(<class 'object_test.ThingWithClassAsDefaultFactory'>, {'A': ThingWithClassAsDefaultFactory(<class 'object_test.ThingWithClassAsDefaultFactory'>, {'B': 1, 'C': 2})})
cls = <class 'object_test.ThingWithClassAsDefaultFactory'>

    def _test_defaultdict_tree(tree, cls):
        tree['A']['B'] = 1
        tree['A']['C'] = 2
        # roundtrip
        encoded = jsonpickle.encode(tree)
        newtree = jsonpickle.decode(encoded)
        # make sure we didn't lose anything
>       assert type(newtree) == cls
E       AssertionError: assert <class 'NoneType'> == <class 'object_test.ThingWithClassAsDefaultFactory'>
E        +  where <class 'NoneType'> = type(None)

tests/object_test.py:690: AssertionError
________________________ test_ordered_dict_unpicklable _________________________

    def test_ordered_dict_unpicklable():
        """Serialize an OrderedDict with unpicklable=False"""
        d = collections.OrderedDict([('c', 3), ('a', 1), ('b', 2)])
        encoded = jsonpickle.encode(d, unpicklable=False)
        decoded = jsonpickle.decode(encoded)
>       assert d == decoded
E       AssertionError: assert OrderedDict([('c', 3), ('a', 1), ('b', 2)]) == None

tests/object_test.py:785: AssertionError
___________________________ test_ordered_dict_nested ___________________________

    def test_ordered_dict_nested():
        """Serialize nested dicts with OrderedDict values"""
        bottom = collections.OrderedDict([('z', 1), ('a', 2)])
        middle = collections.OrderedDict([('c', bottom)])
        top = collections.OrderedDict([('b', middle)])
        encoded = jsonpickle.encode(top)
        decoded = jsonpickle.decode(encoded)
        assert top == decoded
        # test unpicklable=False
        encoded = jsonpickle.encode(top, unpicklable=False)
        decoded = jsonpickle.decode(encoded)
>       assert top == decoded
E       AssertionError: assert OrderedDict([('b', OrderedDict([('c', OrderedDict([('z', 1), ('a', 2)]))]))]) == None

tests/object_test.py:832: AssertionError
_________________________ test_newstyleslots_iterable __________________________

    def test_newstyleslots_iterable():
        """Seriazlie an object with iterable slots"""
        obj = ThingWithIterableSlots('alpha', 'bravo')
        jsonstr = jsonpickle.encode(obj)
        newobj = jsonpickle.decode(jsonstr)
>       assert newobj.a == 'alpha'
E       AttributeError: 'NoneType' object has no attribute 'a'

tests/object_test.py:984: AttributeError
_______________________ MagicTestCase.test_with_pickling _______________________

self = <wizard_test.MagicTestCase testMethod=test_with_pickling>

    def test_with_pickling(self):
        world = World()
        wizard_merlin = Wizard(world, 'Merlin')
        wizard_morgana = Wizard(world, 'Morgana')
        wizard_morgana_prime = Wizard(world, 'Morgana')
    
        self.assertEqual(wizard_morgana.__dict__, wizard_morgana_prime.__dict__)
    
        spell_a = Spell(wizard_merlin, wizard_morgana, 'magic-missile')
        spell_b = Spell(wizard_merlin, wizard_merlin, 'stone-skin')
        spell_c = Spell(wizard_morgana, wizard_merlin, 'geas')
    
        self.assertEqual(wizard_merlin.spells[wizard_morgana][0], spell_a)
        self.assertEqual(wizard_merlin.spells[wizard_merlin][0], spell_b)
        self.assertEqual(wizard_morgana.spells[wizard_merlin][0], spell_c)
        flat_world = encode(world, keys=True)
>       u_world = decode(flat_world, keys=True)

tests/wizard_test.py:162: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
jsonpickle/unpickler.py:88: in decode
    return context.restore(data, reset=reset, classes=classes)
jsonpickle/unpickler.py:362: in restore
    value = self._restore(obj)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:652: in _restore_state
    state = self._restore(obj[tags.STATE])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:798: in _restore_dict
    data[k] = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:431: in _restore_list
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:431: in <listcomp>
    children = [self._restore(v) for v in obj]
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:652: in _restore_state
    state = self._restore(obj[tags.STATE])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:798: in _restore_dict
    data[k] = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:464: in _restore_reduce
    reduce_val = list(map(self._restore, obj[tags.REDUCE]))
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:828: in _restore_tuple
    return tuple([self._restore(v) for v in obj[tags.TUPLE]])
jsonpickle/unpickler.py:828: in <listcomp>
    return tuple([self._restore(v) for v in obj[tags.TUPLE]])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:828: in _restore_tuple
    return tuple([self._restore(v) for v in obj[tags.TUPLE]])
jsonpickle/unpickler.py:828: in <listcomp>
    return tuple([self._restore(v) for v in obj[tags.TUPLE]])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:771: in _restore_object
    return self._restore_object_instance(obj, cls, class_name)
jsonpickle/unpickler.py:747: in _restore_object_instance
    instance = self._restore_object_instance_variables(obj, instance)
jsonpickle/unpickler.py:693: in _restore_object_instance_variables
    instance = self._restore_state(obj, instance)
jsonpickle/unpickler.py:652: in _restore_state
    state = self._restore(obj[tags.STATE])
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:798: in _restore_dict
    data[k] = self._restore(v)
jsonpickle/unpickler.py:344: in _restore
    return restore(obj)
jsonpickle/unpickler.py:513: in _restore_reduce
    stage1.__setitem__(k, v)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <wizard_test.Wizard object at 0x7f269439bb90>

    def __hash__(self):
>       return hash('Wizard %s' % self.name)
E       AttributeError: 'Wizard' object has no attribute 'name'

tests/wizard_test.py:52: AttributeError

----------- coverage: platform linux, python 3.11.0-beta-3 -----------
...

=========================== short test summary info ============================
FAILED tests/collections_test.py::test_dict_no_cycle - AttributeError: 'NoneT...
FAILED tests/collections_test.py::test_dict_self_cycle - KeyError: C(42)
FAILED tests/collections_test.py::test_dict_mutual_cycle - AttributeError: 'N...
FAILED tests/collections_test.py::test_set_self_cycle - assert D(42) in {D(42...
FAILED tests/collections_test.py::test_set_mutual_cycle - assert D(42) in {D(...
FAILED tests/document_test.py::DocumentTestCase::test_cyclical - AttributeErr...
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_class - KeyError: 'name'
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_class_reference - Key...
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_classdict - KeyError:...
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_classlist - KeyError:...
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_type_reference - KeyE...
FAILED tests/jsonpickle_test.py::PicklingTestCase::test_unpickler_on_missing
FAILED tests/jsonpickle_test.py::JSONPickleTestCase::test_dict_subclass - Ass...
FAILED tests/jsonpickle_test.py::JSONPickleTestCase::test_reference_to_list
FAILED tests/jsonpickle_test.py::JSONPickleTestCase::test_refs_recursive - Ty...
FAILED tests/jsonpickle_test.py::PicklingProtocol2TestCase::test_reduce_state_dict
FAILED tests/jsonpickle_test.py::PicklingProtocol2TestCase::test_reduce_state_no_dict
FAILED tests/jsonpickle_test.py::test_repeat_objects_are_expanded - KeyError:...
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_base_object_roundrip
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_dictsubclass - Ass...
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_dictsubclass_notunpickable
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_list_subclass - As...
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_list_subclass_with_data
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_oldstyleclass - Ke...
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_set_subclass - Ass...
FAILED tests/object_test.py::AdvancedObjectsTestCase::test_set_subclass_with_data
FAILED tests/object_test.py::test_defaultdict_roundtrip - TypeError: 'NoneTyp...
FAILED tests/object_test.py::test_defaultdict_roundtrip_simple_lambda - TypeE...
FAILED tests/object_test.py::test_defaultdict_roundtrip_simple_lambda2 - Asse...
FAILED tests/object_test.py::test_defaultdict_and_things_roundtrip_simple_lambda
FAILED tests/object_test.py::test_defaultdict_subclass_with_self_as_default_factory
FAILED tests/object_test.py::test_defaultdict_subclass_with_class_as_default_factory
FAILED tests/object_test.py::test_ordered_dict_unpicklable - AssertionError: ...
FAILED tests/object_test.py::test_ordered_dict_nested - AssertionError: asser...
FAILED tests/object_test.py::test_newstyleslots_iterable - AttributeError: 'N...
FAILED tests/wizard_test.py::MagicTestCase::test_with_pickling - AttributeErr...
ERROR ...
= 36 failed, 214 passed, 45 skipped, 3 deselected, 3 xpassed, 1 error in 4.86s =
ERROR: InvocationError for command .../jsonpickle/.tox/py311/bin/pytest -k 'not (test_thing_with_module or test_thing_with_submodule)' tests (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   py311: commands failed

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (6 by maintainers)

Commits related to this issue

Most upvoted comments

Thank you all for this amazing issue and the accompanying PR! I don’t think I’d have had the time to make this PR myself, as a close family member recently got a bad medical diagnosis. I added a comment in the PR asking if objects serialized with 2.2.0 can be deserialized by 3.0.0, and if not, is there an easy way to implement a helper function to allow that. Apart from that little nitpick, this looks really good! I love the newer CPython versions, especially the performance increases with 3.11+, so I’m glad you guys were testing this.