Skip to content

Commit a955cf2

Browse files
serhiy-storchakaclaude
serhiy-storchaka
and
claude
committed
gh-71549: Add support for JSON serialization of custom types
Add the __json__() and __raw_json__() protocol methods, the copyreg.json() registration function with copyreg.json_dispatch_table, the copyreg.RawJSON wrapper for including already encoded JSON in the output verbatim, and the JSONEncoder.dispatch_table attribute for overriding the global registry per encoder. Make deque, mappingproxy, ChainMap, UserDict, UserList and UserString serializable out of the box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 0023d5b commit a955cf2

20 files changed

Lines changed: 841 additions & 153 deletions

‎Doc/library/collections.rst‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,10 @@ The class can be used to simulate nested scopes and is useful in templating.
5858
one of the underlying mappings gets updated, those changes will be reflected
5959
in :class:`ChainMap`.
6060

61+
.. versionchanged:: next
62+
Added the :meth:`~object.__json__` method;
63+
instances are now serializable by the :mod:`json` module.
64+
6165
All of the usual dictionary methods are supported. In addition, there is a
6266
*maps* attribute, a method for creating new subcontexts, and a property for
6367
accessing all but the first mapping:
@@ -486,6 +490,10 @@ or subtracting from an empty counter.
486490

487491
Deques are :ref:`generic <generics>` over the type of their contents.
488492

493+
.. versionchanged:: next
494+
Added the :meth:`~object.__json__` method;
495+
instances are now serializable by the :mod:`json` module.
496+
489497

490498
Deque objects support the following methods:
491499

@@ -1343,6 +1351,10 @@ attribute.
13431351
:class:`!UserDict` instances. If arguments are provided, they are used to
13441352
initialize :attr:`data`, like a regular dictionary.
13451353

1354+
.. versionchanged:: next
1355+
Added the :meth:`~object.__json__` method;
1356+
instances are now serializable by the :mod:`json` module.
1357+
13461358
In addition to supporting the methods and operations of mappings,
13471359
:class:`!UserDict` instances provide the following attribute:
13481360

@@ -1380,6 +1392,10 @@ to work with because the underlying list is accessible as an attribute.
13801392
defaulting to the empty list ``[]``. *list* can be any iterable, for
13811393
example a real Python list or a :class:`UserList` object.
13821394

1395+
.. versionchanged:: next
1396+
Added the :meth:`~object.__json__` method;
1397+
instances are now serializable by the :mod:`json` module.
1398+
13831399
In addition to supporting the methods and operations of mutable sequences,
13841400
:class:`UserList` instances provide the following attribute:
13851401

@@ -1429,3 +1445,7 @@ attribute.
14291445
.. versionchanged:: 3.5
14301446
New methods ``__getnewargs__``, ``__rmod__``, ``casefold``,
14311447
``format_map``, ``isprintable``, and ``maketrans``.
1448+
1449+
.. versionchanged:: next
1450+
Added the :meth:`~object.__json__` method;
1451+
instances are now serializable by the :mod:`json` module.

‎Doc/library/copyreg.rst‎

Lines changed: 59 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,24 @@
1-
:mod:`!copyreg` --- Register :mod:`!pickle` support functions
2-
=============================================================
1+
:mod:`!copyreg` --- Register :mod:`!pickle` and :mod:`!json` support functions
2+
==============================================================================
33

44
.. module:: copyreg
5-
:synopsis: Register pickle support functions.
5+
:synopsis: Register pickle and JSON support functions.
66

77
**Source code:** :source:`Lib/copyreg.py`
88

99
.. index::
1010
pair: module; pickle
1111
pair: module; copy
12+
pair: module; json
1213

1314
--------------
1415

15-
The :mod:`!copyreg` module offers a way to define functions used while pickling
16-
specific objects. The :mod:`pickle` and :mod:`copy` modules use those functions
17-
when pickling/copying those objects. The module provides configuration
18-
information about object constructors which are not classes.
16+
The :mod:`!copyreg` module offers a way to define functions
17+
used while pickling and JSON-serializing specific objects.
18+
The :mod:`pickle`, :mod:`copy` and :mod:`json` modules use those functions
19+
when pickling/copying/serializing those objects.
20+
The module provides configuration information
21+
about object constructors which are not classes.
1922
Such constructors may be factory functions or class instances.
2023

2124

@@ -39,6 +42,55 @@ Such constructors may be factory functions or class instances.
3942
object or subclass of :class:`pickle.Pickler` can also be used for
4043
declaring reduction functions.
4144

45+
46+
.. function:: json(type, function)
47+
48+
Declares that *function* should be used
49+
as the JSON serialization function for objects of type *type*.
50+
*function* must be callable;
51+
it is called with the object as its only argument
52+
and must return a substitute object to be serialized,
53+
with the same interface as the :meth:`~object.__json__` method.
54+
Registration is by exact type:
55+
it does not apply to subclasses of *type*.
56+
See :ref:`json-protocol` for details.
57+
58+
.. versionadded:: next
59+
60+
61+
.. data:: json_dispatch_table
62+
63+
The mapping of types to serialization functions
64+
filled by :func:`json` and consulted by the :mod:`json` module.
65+
It can be overridden for a particular encoder
66+
with the :attr:`json.JSONEncoder.dispatch_table` attribute.
67+
68+
.. versionadded:: next
69+
70+
71+
.. class:: RawJSON(encoded_json)
72+
73+
Wrapper for the already encoded JSON string *encoded_json*.
74+
The JSON encoder outputs it verbatim,
75+
without validation of its content.
76+
``str()`` of the instance returns *encoded_json*.
77+
78+
For example, it allows serializing :class:`decimal.Decimal`
79+
as a JSON number with full precision:
80+
81+
>>> import copyreg, decimal, json
82+
>>> copyreg.json(decimal.Decimal, lambda d: copyreg.RawJSON(str(d)))
83+
>>> json.dumps({'price': decimal.Decimal('1.10')})
84+
'{"price": 1.10}'
85+
86+
A registration can be undone by removing the entry from
87+
:data:`json_dispatch_table`:
88+
89+
>>> del copyreg.json_dispatch_table[decimal.Decimal]
90+
91+
.. versionadded:: next
92+
93+
4294
Example
4395
-------
4496

‎Doc/library/json.rst‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,11 @@ Basic Usage
243243
.. versionchanged:: 3.6
244244
All optional parameters are now :ref:`keyword-only <keyword-only_parameter>`.
245245

246+
.. versionchanged:: next
247+
Serialization can now be customized per type
248+
with the :meth:`~object.__json__` method and :func:`copyreg.json`.
249+
See :ref:`json-protocol`.
250+
246251

247252
.. function:: dumps(obj, *, skipkeys=False, ensure_ascii=True, \
248253
check_circular=True, allow_nan=True, cls=None, \
@@ -560,6 +565,14 @@ Encoders and Decoders
560565
.. versionchanged:: 3.6
561566
All parameters are now :ref:`keyword-only <keyword-only_parameter>`.
562567

568+
.. versionchanged:: next
569+
The encoder now consults the dispatch table
570+
(:attr:`~JSONEncoder.dispatch_table`
571+
or :data:`copyreg.json_dispatch_table`)
572+
and the :meth:`~object.__json__` method of the object's class
573+
before falling back to *default*.
574+
See :ref:`json-protocol`.
575+
563576

564577
.. method:: default(o)
565578

@@ -598,6 +611,102 @@ Encoders and Decoders
598611
for chunk in json.JSONEncoder().iterencode(bigobject):
599612
mysocket.write(chunk)
600613

614+
.. attribute:: dispatch_table
615+
616+
A mapping of types to serialization functions,
617+
used instead of the global :data:`copyreg.json_dispatch_table`.
618+
It can be set as a class attribute of a subclass
619+
or as an attribute of an encoder instance.
620+
See :ref:`json-protocol`.
621+
622+
.. versionadded:: next
623+
624+
625+
.. _json-protocol:
626+
627+
Serializing custom objects
628+
--------------------------
629+
630+
Serialization of objects of types that are not supported natively
631+
(see the :ref:`conversion table <py-to-json-table>`)
632+
can be customized at three levels:
633+
634+
* The author of a class can define the JSON representation of its instances
635+
by giving it a :meth:`~object.__json__` method.
636+
637+
* An application can register a serialization function
638+
for instances of a type it does not control
639+
with :func:`copyreg.json`.
640+
641+
* A particular call can use its own type-to-function mapping
642+
by setting the :attr:`~JSONEncoder.dispatch_table` attribute
643+
of a :class:`JSONEncoder` subclass or instance.
644+
645+
The more specific level takes precedence:
646+
a registered function is used before ``__json__``,
647+
and an encoder's own dispatch table completely replaces
648+
the global :data:`copyreg.json_dispatch_table`.
649+
The *default* function is called last,
650+
only for objects that none of the above handled.
651+
652+
.. method:: object.__json__()
653+
654+
Return a substitute object to be serialized instead of *self*.
655+
Called by the JSON encoder for an object
656+
whose type is not supported natively
657+
and has no entry in the dispatch table.
658+
659+
If the result is of a serializable type, it is serialized as usual
660+
(but a ``__json__`` method of the result is not consulted).
661+
Otherwise the result is interpreted by duck typing:
662+
663+
* an object with :meth:`~object.__index__` is serialized
664+
as a JSON number;
665+
* an object with :meth:`~object.__float__` is serialized
666+
as a JSON number;
667+
* an iterable with a :meth:`!keys` method and
668+
:meth:`~object.__getitem__` is serialized as a JSON object;
669+
* any other iterable is serialized as a JSON array;
670+
* an object with :meth:`~object.__raw_json__` is included
671+
in the output verbatim.
672+
673+
.. versionadded:: next
674+
675+
.. method:: object.__raw_json__()
676+
677+
Return a string to be included in the JSON output verbatim,
678+
without validation of its content.
679+
Consulted only for the result of a registered serialization function
680+
or a :meth:`~object.__json__` method.
681+
To include an already encoded fragment directly in the serialized data,
682+
wrap it in :class:`copyreg.RawJSON` instead.
683+
684+
.. versionadded:: next
685+
686+
For example, a class can serialize itself as a JSON object::
687+
688+
class Point:
689+
def __init__(self, x, y):
690+
self.x = x
691+
self.y = y
692+
693+
def __json__(self):
694+
return {'x': self.x, 'y': self.y}
695+
696+
An application can choose how :class:`decimal.Decimal` is serialized —
697+
as a JSON string::
698+
699+
copyreg.json(decimal.Decimal, str)
700+
701+
or as a JSON number with full precision, using :class:`copyreg.RawJSON`::
702+
703+
copyreg.json(decimal.Decimal, lambda d: copyreg.RawJSON(str(d)))
704+
705+
A number of standard library container types define
706+
:meth:`~object.__json__` and are therefore serializable out of the box.
707+
708+
.. versionadded:: next
709+
601710

602711
Exceptions
603712
----------

‎Doc/library/types.rst‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -397,6 +397,10 @@ Standard names are defined for the following types:
397397
Updated to support the new union (``|``) operator from :pep:`584`, which
398398
simply delegates to the underlying mapping.
399399

400+
.. versionchanged:: next
401+
Added the :meth:`~object.__json__` method;
402+
instances are now serializable by the :mod:`json` module.
403+
400404
.. describe:: key in proxy
401405

402406
Return ``True`` if the underlying mapping has a key *key*, else

‎Doc/whatsnew/3.16.rst‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,16 @@ codecs
9696
even when a built-in codec of the same name exists.
9797
(Contributed by Serhiy Storchaka in :gh:`152997`.)
9898

99+
copyreg
100+
-------
101+
102+
* Add :func:`copyreg.json`, :data:`copyreg.json_dispatch_table`
103+
and :class:`copyreg.RawJSON`
104+
for registering JSON serialization functions
105+
for types that cannot be modified;
106+
see the :mod:`json` entry below.
107+
(Contributed by Serhiy Storchaka in :gh:`71549`.)
108+
99109
curses
100110
------
101111

@@ -293,6 +303,30 @@ Add :meth:`~ipaddress.IPv4Network.next_network` and
293303
network with a specific prefix size.
294304

295305

306+
json
307+
----
308+
309+
* JSON serialization of custom types can now be customized
310+
without subclassing :class:`~json.JSONEncoder` or passing *default*:
311+
a class can define its JSON representation
312+
with the new :meth:`~object.__json__` method,
313+
an application can register a serialization function
314+
for a type it does not control with the new :func:`copyreg.json`,
315+
and a particular encoder can use its own type-to-function mapping
316+
via the new :attr:`~json.JSONEncoder.dispatch_table` attribute.
317+
The new :class:`copyreg.RawJSON` wrapper includes
318+
an already encoded JSON string in the output verbatim,
319+
which allows, for example, serializing :class:`decimal.Decimal`
320+
as a JSON number with full precision.
321+
Standard library containers with an unambiguous JSON form
322+
(:class:`collections.deque`, :class:`types.MappingProxyType`,
323+
:class:`collections.ChainMap`, :class:`collections.UserDict`,
324+
:class:`collections.UserList` and :class:`collections.UserString`)
325+
are now serializable out of the box.
326+
See :ref:`json-protocol`.
327+
(Contributed by Serhiy Storchaka in :gh:`71549`.)
328+
329+
296330
logging
297331
-------
298332

‎Include/internal/pycore_global_objects_fini_generated.h‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_global_strings.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,7 @@ struct _Py_global_strings {
169169
STRUCT_FOR_ID(__iter__)
170170
STRUCT_FOR_ID(__itruediv__)
171171
STRUCT_FOR_ID(__ixor__)
172+
STRUCT_FOR_ID(__json__)
172173
STRUCT_FOR_ID(__lazy_import__)
173174
STRUCT_FOR_ID(__lazy_modules__)
174175
STRUCT_FOR_ID(__le__)
@@ -206,6 +207,7 @@ struct _Py_global_strings {
206207
STRUCT_FOR_ID(__qualname__)
207208
STRUCT_FOR_ID(__radd__)
208209
STRUCT_FOR_ID(__rand__)
210+
STRUCT_FOR_ID(__raw_json__)
209211
STRUCT_FOR_ID(__rdivmod__)
210212
STRUCT_FOR_ID(__reduce__)
211213
STRUCT_FOR_ID(__reduce_ex__)

‎Include/internal/pycore_runtime_init_generated.h‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Include/internal/pycore_unicodeobject_generated.h‎

Lines changed: 8 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)