Skip to content

Commit 02c9d3a

Browse files
zzzeekzzzeek
committed
unmapped_dataclass
Reworked the handling of classes which extend from MappedAsDataclass but are not themselves mapped, i.e. the declarative base as well as any mixins or abstract classes. These classes as before are turned into real dataclasses, however a scan now takes place across the mapped elements such as mapped_column(), relationship(), etc. so that we may also take into account dataclasses.field-specific parameters like init=False, repr, etc. The main use case for this is so that mixin dataclasses may make use of "default" in fields while not being rejected by the dataclasses constructor. The generated classes are more functional as dataclasses in a standalone fashion as well, even though this is not their intended use. As a standalone dataclass, the one feature that does not work is a field that has a default with init=False, because we still need to have a mapped_column() or similar present at the class level for the class to work as a superclass. The change also addes the :func:`_orm.unmapped_dataclass` decorator function, which may be used to create unmapped superclasses in a mapped hierarchy that is using the :func:`_orm.mapped_dataclass` decorator to create mapped dataclasses. Previously there was no way to use unmapped dataclass mixins with the decorator approach. Finally, the warning added in 2.0 for 🎫`9350` is turned into an error as mentioned for 2.1, since we're deep into dataclass hierarchy changes here. Fixes: #12854 Change-Id: I11cd8c628d49e9ff1bdbda8a09f4112b40d84be7
1 parent 8bb8c1a commit 02c9d3a

18 files changed

Lines changed: 1777 additions & 947 deletions

File tree

‎doc/build/changelog/unreleased_21/12854.rst‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
.. change::
2+
:tags: usecase, orm
3+
:tickets: 12854
4+
5+
Improvements to the use case of using :ref:`Declarative Dataclass Mapping
6+
<orm_declarative_native_dataclasses>` with intermediary classes that are
7+
unmapped. As was the existing behavior, classes can subclass
8+
:class:`_orm.MappedAsDataclass` alone without a declarative base to act as
9+
mixins, or along with a declarative base as well as ``__abstract__ = True``
10+
to define an abstract base. However, the improved behavior scans ORM
11+
attributes like :func:`_orm.mapped_column` in this case to create correct
12+
``dataclasses.field()`` constructs based on their arguments, allowing for
13+
more natural ordering of fields without dataclass errors being thrown.
14+
Additionally, added a new :func:`_orm.unmapped_dataclass` decorator
15+
function, which may be used to create unmapped mixins in a mapped hierarchy
16+
that is using the :func:`_orm.mapped_dataclass` decorator to create mapped
17+
dataclasses.
18+
19+
.. seealso::
20+
21+
:ref:`orm_declarative_dc_mixins`

‎doc/build/errors.rst‎

Lines changed: 49 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1397,14 +1397,13 @@ notes at :ref:`migration_20_step_six` for an example.
13971397
When transforming <cls> to a dataclass, attribute(s) originate from superclass <cls> which is not a dataclass.
13981398
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13991399

1400-
This warning occurs when using the SQLAlchemy ORM Mapped Dataclasses feature
1400+
This error occurs when using the SQLAlchemy ORM Mapped Dataclasses feature
14011401
described at :ref:`orm_declarative_native_dataclasses` in conjunction with
14021402
any mixin class or abstract base that is not itself declared as a
14031403
dataclass, such as in the example below::
14041404

14051405
from __future__ import annotations
14061406

1407-
import inspect
14081407
from typing import Optional
14091408
from uuid import uuid4
14101409

@@ -1434,18 +1433,17 @@ dataclass, such as in the example below::
14341433
email: Mapped[str] = mapped_column()
14351434

14361435
Above, since ``Mixin`` does not itself extend from :class:`_orm.MappedAsDataclass`,
1437-
the following warning is generated:
1436+
the following error is generated:
14381437

14391438
.. sourcecode:: none
14401439

1441-
SADeprecationWarning: When transforming <class '__main__.User'> to a
1442-
dataclass, attribute(s) "create_user", "update_user" originates from
1443-
superclass <class
1444-
'__main__.Mixin'>, which is not a dataclass. This usage is deprecated and
1445-
will raise an error in SQLAlchemy 2.1. When declaring SQLAlchemy
1446-
Declarative Dataclasses, ensure that all mixin classes and other
1447-
superclasses which include attributes are also a subclass of
1448-
MappedAsDataclass.
1440+
sqlalchemy.exc.InvalidRequestError: When transforming <class
1441+
'__main__.User'> to a dataclass, attribute(s) 'create_user', 'update_user'
1442+
originates from superclass <class '__main__.Mixin'>, which is not a
1443+
dataclass. When declaring SQLAlchemy Declarative Dataclasses, ensure that
1444+
all mixin classes and other superclasses which include attributes are also
1445+
a subclass of MappedAsDataclass or make use of the @unmapped_dataclass
1446+
decorator.
14491447

14501448
The fix is to add :class:`_orm.MappedAsDataclass` to the signature of
14511449
``Mixin`` as well::
@@ -1454,6 +1452,41 @@ The fix is to add :class:`_orm.MappedAsDataclass` to the signature of
14541452
create_user: Mapped[int] = mapped_column()
14551453
update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False)
14561454

1455+
When using decorators like :func:`_orm.mapped_as_dataclass` to map, the
1456+
:func:`_orm.unmapped_dataclass` may be used to indicate mixins::
1457+
1458+
from __future__ import annotations
1459+
1460+
from typing import Optional
1461+
from uuid import uuid4
1462+
1463+
from sqlalchemy import String
1464+
from sqlalchemy.orm import Mapped
1465+
from sqlalchemy.orm import mapped_as_dataclass
1466+
from sqlalchemy.orm import mapped_column
1467+
from sqlalchemy.orm import registry
1468+
from sqlalchemy.orm import unmapped_dataclass
1469+
1470+
1471+
@unmapped_dataclass
1472+
class Mixin:
1473+
create_user: Mapped[int] = mapped_column()
1474+
update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False)
1475+
1476+
1477+
reg = registry()
1478+
1479+
1480+
@mapped_as_dataclass(reg)
1481+
class User(Mixin):
1482+
__tablename__ = "sys_user"
1483+
1484+
uid: Mapped[str] = mapped_column(
1485+
String(50), init=False, default_factory=uuid4, primary_key=True
1486+
)
1487+
username: Mapped[str] = mapped_column()
1488+
email: Mapped[str] = mapped_column()
1489+
14571490
Python's :pep:`681` specification does not accommodate for attributes declared
14581491
on superclasses of dataclasses that are not themselves dataclasses; per the
14591492
behavior of Python dataclasses, such fields are ignored, as in the following
@@ -1482,14 +1515,12 @@ Above, the ``User`` class will not include ``create_user`` in its constructor
14821515
nor will it attempt to interpret ``update_user`` as a dataclass attribute.
14831516
This is because ``Mixin`` is not a dataclass.
14841517

1485-
SQLAlchemy's dataclasses feature within the 2.0 series does not honor this
1486-
behavior correctly; instead, attributes on non-dataclass mixins and
1487-
superclasses are treated as part of the final dataclass configuration. However
1488-
type checkers such as Pyright and Mypy will not consider these fields as
1489-
part of the dataclass constructor as they are to be ignored per :pep:`681`.
1490-
Since their presence is ambiguous otherwise, SQLAlchemy 2.1 will require that
1518+
Since type checkers such as Pyright and Mypy will not consider these fields as
1519+
part of the dataclass constructor as they are to be ignored per :pep:`681`,
1520+
their presence becomes ambiguous. Therefore SQLAlchemy requires that
14911521
mixin classes which have SQLAlchemy mapped attributes within a dataclass
1492-
hierarchy have to themselves be dataclasses.
1522+
hierarchy have to themselves be dataclasses using SQLAlchemy's unmapped
1523+
dataclass feature.
14931524

14941525

14951526
.. _error_dcte:

‎doc/build/orm/dataclasses.rst‎

Lines changed: 94 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,8 @@ decorator.
5252
Dataclass conversion may be added to any Declarative class either by adding the
5353
:class:`_orm.MappedAsDataclass` mixin to a :class:`_orm.DeclarativeBase` class
5454
hierarchy, or for decorator mapping by using the
55-
:meth:`_orm.registry.mapped_as_dataclass` class decorator.
55+
:meth:`_orm.registry.mapped_as_dataclass` class decorator or its
56+
functional variant :func:`_orm.mapped_as_dataclass`.
5657

5758
The :class:`_orm.MappedAsDataclass` mixin may be applied either
5859
to the Declarative ``Base`` class or any superclass, as in the example
@@ -231,13 +232,14 @@ and ``fullname`` is optional. The ``id`` field, which we expect to be
231232
database-generated, is not part of the constructor at all::
232233

233234
from sqlalchemy.orm import Mapped
235+
from sqlalchemy.orm import mapped_as_dataclass
234236
from sqlalchemy.orm import mapped_column
235237
from sqlalchemy.orm import registry
236238

237239
reg = registry()
238240

239241

240-
@reg.mapped_as_dataclass
242+
@mapped_as_dataclass(reg)
241243
class User:
242244
__tablename__ = "user_account"
243245

@@ -268,13 +270,14 @@ but where the parameter is optional in the constructor::
268270

269271
from sqlalchemy import func
270272
from sqlalchemy.orm import Mapped
273+
from sqlalchemy.orm import mapped_as_dataclass
271274
from sqlalchemy.orm import mapped_column
272275
from sqlalchemy.orm import registry
273276

274277
reg = registry()
275278

276279

277-
@reg.mapped_as_dataclass
280+
@mapped_as_dataclass(reg)
278281
class User:
279282
__tablename__ = "user_account"
280283

@@ -323,6 +326,7 @@ emit a deprecation warning::
323326
from typing import Annotated
324327

325328
from sqlalchemy.orm import Mapped
329+
from sqlalchemy.orm import mapped_as_dataclass
326330
from sqlalchemy.orm import mapped_column
327331
from sqlalchemy.orm import registry
328332

@@ -332,7 +336,7 @@ emit a deprecation warning::
332336
reg = registry()
333337

334338

335-
@reg.mapped_as_dataclass
339+
@mapped_as_dataclass(reg)
336340
class User:
337341
__tablename__ = "user_account"
338342
id: Mapped[intpk]
@@ -348,6 +352,7 @@ the other arguments can remain within the ``Annotated`` construct::
348352
from typing import Annotated
349353

350354
from sqlalchemy.orm import Mapped
355+
from sqlalchemy.orm import mapped_as_dataclass
351356
from sqlalchemy.orm import mapped_column
352357
from sqlalchemy.orm import registry
353358

@@ -356,7 +361,7 @@ the other arguments can remain within the ``Annotated`` construct::
356361
reg = registry()
357362

358363

359-
@reg.mapped_as_dataclass
364+
@mapped_as_dataclass(reg)
360365
class User:
361366
__tablename__ = "user_account"
362367

@@ -371,15 +376,19 @@ the other arguments can remain within the ``Annotated`` construct::
371376
Using mixins and abstract superclasses
372377
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
373378

374-
Any mixins or base classes that are used in a :class:`_orm.MappedAsDataclass`
375-
mapped class which include :class:`_orm.Mapped` attributes must themselves be
376-
part of a :class:`_orm.MappedAsDataclass`
377-
hierarchy, such as in the example below using a mixin::
379+
Mixin and abstract superclass are supported with the Declarative Dataclass
380+
Mapping by defining classes that are part of the :class:`_orm.MappedAsDataclass`
381+
hierarchy, either without including a declarative base or by setting
382+
``__abstract__ = True``. The example below illustrates a class ``Mixin`` that is
383+
not itself mapped, but serves as part of the base for a mapped class::
384+
385+
from sqlalchemy.orm import DeclarativeBase
386+
from sqlalchemy.orm import MappedAsDataclass
378387

379388

380389
class Mixin(MappedAsDataclass):
381390
create_user: Mapped[int] = mapped_column()
382-
update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False)
391+
update_user: Mapped[Optional[int]] = mapped_column(default=None)
383392

384393

385394
class Base(DeclarativeBase, MappedAsDataclass):
@@ -395,21 +404,77 @@ hierarchy, such as in the example below using a mixin::
395404
username: Mapped[str] = mapped_column()
396405
email: Mapped[str] = mapped_column()
397406

398-
Python type checkers which support :pep:`681` will otherwise not consider
399-
attributes from non-dataclass mixins to be part of the dataclass.
407+
.. tip::
400408

401-
.. deprecated:: 2.0.8 Using mixins and abstract bases within
402-
:class:`_orm.MappedAsDataclass` or
403-
:meth:`_orm.registry.mapped_as_dataclass` hierarchies which are not
404-
themselves dataclasses is deprecated, as these fields are not supported
405-
by :pep:`681` as belonging to the dataclass. A warning is emitted for this
406-
case which will later be an error.
409+
When using :class:`_orm.MappedAsDataclass` without a declarative base in
410+
the hiearchy, the target class is still turned into a real Python dataclass,
411+
so that it may properly serve as a base for a mapped dataclass. Using
412+
:class:`_orm.MappedAsDataclass` (or the :func:`_orm.unmapped_dataclass` decorator
413+
described later in this section) is required in order for the class to be correctly
414+
recognized by type checkers as SQLAlchemy-enabled dataclasses. Declarative
415+
itself will reject mixins / abstract classes that are not themselves
416+
Declarative Dataclasses (e.g. they can't be plain classes nor can they be
417+
plain ``@dataclass`` classes).
407418

408-
.. seealso::
419+
.. seealso::
409420

410-
:ref:`error_dcmx` - background on rationale
421+
:ref:`error_dcmx` - further background
411422

423+
Another example, where an abstract base combines :class:`_orm.MappedAsDataclass`
424+
with ``__abstract__ = True``::
412425

426+
from sqlalchemy.orm import DeclarativeBase
427+
from sqlalchemy.orm import MappedAsDataclass
428+
429+
430+
class Base(DeclarativeBase, MappedAsDataclass):
431+
pass
432+
433+
434+
class AbstractUser(Base):
435+
__abstract__ = True
436+
437+
create_user: Mapped[int] = mapped_column()
438+
update_user: Mapped[Optional[int]] = mapped_column(default=None)
439+
440+
441+
class User(AbstractUser):
442+
__tablename__ = "sys_user"
443+
444+
uid: Mapped[str] = mapped_column(
445+
String(50), init=False, default_factory=uuid4, primary_key=True
446+
)
447+
username: Mapped[str] = mapped_column()
448+
email: Mapped[str] = mapped_column()
449+
450+
Finally, for a hierarchy that's based on use of the :func:`_orm.mapped_as_dataclass`
451+
decorator, mixins may be defined using the :func:`_orm.unmapped_dataclass` decorator::
452+
453+
from sqlalchemy.orm import registry
454+
from sqlalchemy.orm import mapped_as_dataclass
455+
from sqlalchemy.orm import unmapped_dataclass
456+
457+
458+
@unmapped_dataclass()
459+
class Mixin:
460+
create_user: Mapped[int] = mapped_column()
461+
update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False)
462+
463+
464+
reg = registry()
465+
466+
467+
@mapped_as_dataclass(reg)
468+
class User(Mixin):
469+
__tablename__ = "sys_user"
470+
471+
uid: Mapped[str] = mapped_column(
472+
String(50), init=False, default_factory=uuid4, primary_key=True
473+
)
474+
username: Mapped[str] = mapped_column()
475+
email: Mapped[str] = mapped_column()
476+
477+
.. versionadded:: 2.1 Added :func:`_orm.unmapped_dataclass`
413478

414479
.. _orm_declarative_dc_relationships:
415480

@@ -429,14 +494,15 @@ scalar object references may make use of
429494

430495
from sqlalchemy import ForeignKey
431496
from sqlalchemy.orm import Mapped
497+
from sqlalchemy.orm import mapped_as_dataclass
432498
from sqlalchemy.orm import mapped_column
433499
from sqlalchemy.orm import registry
434500
from sqlalchemy.orm import relationship
435501

436502
reg = registry()
437503

438504

439-
@reg.mapped_as_dataclass
505+
@mapped_as_dataclass(reg)
440506
class Parent:
441507
__tablename__ = "parent"
442508
id: Mapped[int] = mapped_column(primary_key=True)
@@ -445,7 +511,7 @@ scalar object references may make use of
445511
)
446512

447513

448-
@reg.mapped_as_dataclass
514+
@mapped_as_dataclass(reg)
449515
class Child:
450516
__tablename__ = "child"
451517
id: Mapped[int] = mapped_column(primary_key=True)
@@ -478,13 +544,14 @@ of the object, but will not be persisted by the ORM::
478544

479545

480546
from sqlalchemy.orm import Mapped
547+
from sqlalchemy.orm import mapped_as_dataclass
481548
from sqlalchemy.orm import mapped_column
482549
from sqlalchemy.orm import registry
483550

484551
reg = registry()
485552

486553

487-
@reg.mapped_as_dataclass
554+
@mapped_as_dataclass(reg)
488555
class Data:
489556
__tablename__ = "data"
490557

@@ -513,13 +580,14 @@ function, such as `bcrypt <https://pypi.org/project/bcrypt/>`_ or
513580
from typing import Optional
514581

515582
from sqlalchemy.orm import Mapped
583+
from sqlalchemy.orm import mapped_as_dataclass
516584
from sqlalchemy.orm import mapped_column
517585
from sqlalchemy.orm import registry
518586

519587
reg = registry()
520588

521589

522-
@reg.mapped_as_dataclass
590+
@mapped_as_dataclass(reg)
523591
class User:
524592
__tablename__ = "user_account"
525593

@@ -571,7 +639,8 @@ Integrating with Alternate Dataclass Providers such as Pydantic
571639
details which **explicitly resolve** these incompatibilities.
572640

573641
SQLAlchemy's :class:`_orm.MappedAsDataclass` class
574-
and :meth:`_orm.registry.mapped_as_dataclass` method call directly into
642+
:meth:`_orm.registry.mapped_as_dataclass` method, and
643+
:func:`_orm.mapped_as_dataclass` functions call directly into
575644
the Python standard library ``dataclasses.dataclass`` class decorator, after
576645
the declarative mapping process has been applied to the class. This
577646
function call may be swapped out for alternateive dataclasses providers,

‎doc/build/orm/mapping_api.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,4 +146,6 @@ Class Mapping API
146146

147147
.. autofunction:: synonym_for
148148

149+
.. autofunction:: unmapped_dataclass
150+
149151

0 commit comments

Comments
 (0)