marshmallow-sqlalchemy: Conversion errors with `SQLAlchemy==2.0.2`
Description
It looks like ma-sqla doesn’t work with the newest version of SQLAlchemy==2.0.2. Here is an example taken from docs compiled into a single file:
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker, relationship, backref
from marshmallow_sqlalchemy import SQLAlchemySchema, auto_field
engine = sa.create_engine("sqlite:///:memory:")
session = scoped_session(sessionmaker(bind=engine))
Base = declarative_base()
class Author(Base):
__tablename__ = "authors"
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String, nullable=False)
def __repr__(self):
return "<Author(name={self.name!r})>".format(self=self)
class Book(Base):
__tablename__ = "books"
id = sa.Column(sa.Integer, primary_key=True)
title = sa.Column(sa.String)
author_id = sa.Column(sa.Integer, sa.ForeignKey("authors.id"))
author = relationship("Author", backref=backref("books"))
Base.metadata.create_all(engine)
class AuthorSchema(SQLAlchemySchema):
class Meta:
model = Author
load_instance = True # Optional: deserialize to model instances
id = auto_field()
name = auto_field()
books = auto_field()
class BookSchema(SQLAlchemySchema):
class Meta:
model = Book
load_instance = True
id = auto_field()
title = auto_field()
author_id = auto_field()
When I am using SQLAlchemy==2.0.1 everything works, with a minor warning. But when I am update to SQLAlchemy==2.0.2 I get the following error:
../test.py:8: MovedIn20Warning: The ``declarative_base()`` function is now available as sqlalchemy.orm.declarative_base(). (deprecated since: 2.0) (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
Base = declarative_base()
Traceback (most recent call last):
File "../test.py", line 31, in <module>
class AuthorSchema(SQLAlchemySchema):
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow/schema.py", line 121, in __new__
klass._declared_fields = mcs.get_declared_fields(
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow_sqlalchemy/schema.py", line 92, in get_declared_fields
fields.update(mcs.get_auto_fields(fields, converter, opts, dict_cls))
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow_sqlalchemy/schema.py", line 102, in get_auto_fields
{
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow_sqlalchemy/schema.py", line 103, in <dictcomp>
field_name: field.create_field(
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow_sqlalchemy/schema.py", line 28, in create_field
return converter.field_for(model, column_name, **self.field_kwargs)
File ".pyenv/versions/elan/lib/python3.10/site-packages/marshmallow_sqlalchemy/convert.py", line 222, in field_for
attr = getattr(model, property_name)
AttributeError: type object 'Author' has no attribute 'books'
It looks like heuristics in marshmallow_sqlalchemy/convert.py don’t correspond to the SQLAlchemy in the new version. I noticed it for my personal use case (I am not showing it here) fails to detect relationship during the conversion
Env
It was spotted in python:3.10 the requirements are:
marshmallow-sqlalchemy==0.28.1
SQLAlchemy==2.0.2
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 23 (8 by maintainers)
@lafrech
The following https://github.com/marshmallow-code/marshmallow-sqlalchemy/pull/495 fixes the issue with the lack of implicit calls to registry.configure() for both iterate_properties and get_property. It now passes all tests.
Feel free to take a look – and also unless there are any other compatibility issues with SQLAlchemy 2.0, perhaps we can also adjust the requirements as well. I haven’t tested anything beyond the unit tests already available.
The configure_mappers() should be called after importing SQLAlchemy models, where the relationships are defined, but before the Schemas are imported.
In my particular setup, I have a models folder and a schema folders and things get initialized in that order:
models/ init.py -> this will import all the models from each of the models file, I have ~ 1 file per table. Once the models are imported, it will import schemas. (see init below)
schemas/ init.py -> This will first configure mappers at the top, then import all the schemas from their respective files.
Regardless if your file structure, calling configure_mappers() before the models are imported will not work.
Oh, then it was a pure luck that it worked for my setup 😅
@lafrech many thanks for taking initiative for that. Je te souhaite bonne chance 🤞
Oh, right. Thanks for the feedback, then. Release should come soon.
Thanks @ddoyon92.
I was wrong in my comment: the method we used was not underscored. It’s strange that it was removed but there must be a rationale for it. Anyway, following the advice in the changelog and using
.attrsis the way to go.I’ll review and merge.
We may want to double-check the migration guide before finalizing a SQlAlchemy 2.x compatible release.
I’ve tried this before, it works fine as it does an implicit call to registry.configure(). However, it may be more a patch than anything, and may break again in the future. That being said, it definitely works.
It is equivalent to my suggestion above of calling configure_mappers(), but implicitly. For Marshmallow users, that is definitely a better patch. We do need to keep in mind that if SQLAlchemy phased that one out, we may eventually run into more issues, but we can kick the bucket down the road.
I can submit a PR for this if we want to go along with it. It’s a very minor modification.
Splitting model from schema’s and calling configure_mappers(); worked for me
Rhynzler above is right.
I have faced also issues with relationships due to this. For anyone else facing this issue, for now my cleanest option has been to call configure_mappers() before importing my schemas.
from sqlalchemy.orm import configure_mappersconfigure_mappers()