|
| 1 | +# -*- coding: utf-8; -*- |
| 2 | +"""A pickle-aware singleton abstraction. |
| 3 | +
|
| 4 | +To use it, inherit your class from `Singleton`. Can be used as a mixin. |
| 5 | +
|
| 6 | +
|
| 7 | +**Behavior** |
| 8 | +
|
| 9 | +- If you have instantiated a singleton, and then unpickle an instance of that same |
| 10 | + singleton type, the object identity will not change. When the unpickling procedure |
| 11 | + attempts to create the instance, it will get redirected to the existing instance. |
| 12 | +
|
| 13 | + This allows object identity checks to identify the singletons across pickle dumps |
| 14 | + (e.g. there is only one linked list terminator `nil` in the process, whether or not |
| 15 | + some linked lists were loaded from a pickle dump). |
| 16 | +
|
| 17 | +
|
| 18 | +- Upon unpickling, by default, any existing instance data the singleton instance has |
| 19 | + is overwritten with the instance data from the pickle dump. |
| 20 | +
|
| 21 | + This is due to the behavior of the default `__setstate__`, but it is also the |
| 22 | + solution of least surprise. Arguably this is exactly the expected behavior: |
| 23 | + there's only one instance of the singleton, and its state is restored upon |
| 24 | + unpickling. |
| 25 | +
|
| 26 | + If you want something else, what happens to the pickled instance data can |
| 27 | + be customized in the standard pythonic way, with custom `__getstate__` and |
| 28 | + `__setstate__` methods in your class definition. |
| 29 | +
|
| 30 | + https://docs.python.org/3/library/pickle.html#object.__getstate__ |
| 31 | + https://docs.python.org/3/library/pickle.html#object.__setstate__ |
| 32 | +
|
| 33 | +- We expect you to manage references to singleton instances manually, just like |
| 34 | + for regular objects. Calling the constructor of a singleton type again while |
| 35 | + an instance still exists is considered a `TypeError` (since it's a type that |
| 36 | + doesn't support that operation). |
| 37 | +
|
| 38 | + This break from the tradition of the classical singleton design pattern allows us |
| 39 | + to better adhere to the principles of fail-fast and least surprise, as well as |
| 40 | + separate the concerns of enforcing singleton-ness and obtaining the instance |
| 41 | + reference, all of which arguably makes this solution more pythonic. |
| 42 | +
|
| 43 | +
|
| 44 | +Note this is a true singleton, not a Borg; there is really only one instance. |
| 45 | +""" |
| 46 | + |
| 47 | +# **Technical details** |
| 48 | +# |
| 49 | +# To make this work: |
| 50 | +# |
| 51 | +# 1) We store the references to the singleton instances in a variable outside |
| 52 | +# any class, at the module top level. |
| 53 | +# |
| 54 | +# If the `_instances` dictionary was stored in the class or in the metaclass, |
| 55 | +# it would get clobbered at unpickle time, leading to a situation where the |
| 56 | +# existing singleton instance (if someone has kept a reference to it) and the |
| 57 | +# unpickled instance are different. Obviously, for singletons we don't want |
| 58 | +# that - it should be the same instance whether or not it came from a pickle dump. |
| 59 | +# |
| 60 | +# Keeping the `_instances` dictionary external to the singletons themselves, |
| 61 | +# the currently existing singleton instance will prevail even when a singleton |
| 62 | +# is unpickled into a process that already has an instance of that particular |
| 63 | +# singleton. |
| 64 | +# |
| 65 | +# This module only keeps weak references to the singleton instances, so when |
| 66 | +# your last reference to a singleton instance is deleted, that singleton |
| 67 | +# instance becomes eligible for garbage collection. |
| 68 | +# |
| 69 | +# (Once the instance is actually destroyed, the machinery will know it, so |
| 70 | +# you can then create a new instance (again, exactly one!) of that type of |
| 71 | +# singleton, if you want. Leaving aside the question whether there are valid |
| 72 | +# use cases for this behavior, it is arguably what is expected.) |
| 73 | +# |
| 74 | +# 2) Instance creation is customized in two layers: |
| 75 | +# |
| 76 | +# - Metaclass, which intercepts constructor invocations, and arranges things |
| 77 | +# so that if the singleton instance for a particular class already exists, |
| 78 | +# invoking the constructor again raises `TypeError`. |
| 79 | +# |
| 80 | +# - Base class, which customizes **instance creation** proper, with a custom |
| 81 | +# `__new__`. This actually manages the single instance. |
| 82 | +# |
| 83 | +# This separation is necessary, because pickle does not call the class at |
| 84 | +# unpickling time. Hence, when unpickling, our custom metaclass has no |
| 85 | +# chance to act; only `__new__` (of the instance's class) and `__setstate__` |
| 86 | +# (of the instance) are called. |
| 87 | +# |
| 88 | +# There is an inherent impedance mismatch between singleton semantics and |
| 89 | +# the language allowing to express the creation of multiple instances of |
| 90 | +# the same class (which, with the exception of singletons, is always The |
| 91 | +# Right Thing). |
| 92 | +# |
| 93 | +# This leads to a design choice, with three options as to what to do when |
| 94 | +# the constructor of a singleton is called second and further times: |
| 95 | +# |
| 96 | +# a) Let `__init__` re-run each time, overwriting the state. Surprising, |
| 97 | +# and particularly bad, because an innocent-looking constructor call |
| 98 | +# will magically mutate existing state. Good luck tracking down any bugs. |
| 99 | +# |
| 100 | +# b) Let `__init__` run only once, but allow using the constructor call as |
| 101 | +# a shorthand to get the singleton instance, also when it already exists. |
| 102 | +# This perhaps best mimics the classical singleton design pattern. But this |
| 103 | +# behavior can still lead to surprises, because the new state provided to |
| 104 | +# the second and later constructor calls doesn't take. Probably slightly |
| 105 | +# easier to debug than the first option, though. |
| 106 | +# |
| 107 | +# c) Make second and further constructor calls raise `TypeError`, triggering |
| 108 | +# an explicit error as early as possible. |
| 109 | +# |
| 110 | +# We have chosen c). |
| 111 | + |
| 112 | +__all__ = ["Singleton"] |
| 113 | + |
| 114 | +from weakref import WeakValueDictionary |
| 115 | + |
| 116 | +_instances = WeakValueDictionary() |
| 117 | + |
| 118 | +# Metaclass: override default __new__ + __init__ behavior, to allow creating |
| 119 | +# at most one instance. |
| 120 | +# |
| 121 | +# Because magic methods are looked up **on the type**, not on the instance, |
| 122 | +# we override `__call__` **in the metaclass**, in order to override calls |
| 123 | +# of the class (i.e. constructor invocations). |
| 124 | +class ThereCanBeOnlyOne(type): |
| 125 | + def __call__(cls, *args, **kwargs): |
| 126 | + # Here we can do things like call `cls.__init__` only if `cls` was not |
| 127 | + # already in `_instances`... or outright refuse to create a second instance: |
| 128 | + if cls in _instances: |
| 129 | + raise TypeError("Singleton instance of {} already exists".format(cls)) |
| 130 | + # When allowed to proceed, we mimic default behavior. |
| 131 | + # TODO: Maybe we should just "return super().__call__(cls, *args, **kwargs)"? |
| 132 | + # TODO: That doesn't work in the case where we have extra arguments, |
| 133 | + # TODO: so for now we do this manually. Maybe investigate later. |
| 134 | + instance = cls.__new__(cls, *args, **kwargs) |
| 135 | + cls.__init__(instance, *args, **kwargs) |
| 136 | + return instance |
| 137 | + |
| 138 | +# Base class: override instance creation, in a way that interacts correctly |
| 139 | +# with pickle. |
| 140 | +# |
| 141 | +# A base class does that fine, a metaclass by itself doesn't. The class won't |
| 142 | +# get called at unpickle time, so the metaclass's `__call__` has no chance to |
| 143 | +# act. Also, since we want to customize *instance creation*, not change the |
| 144 | +# semantics of the class definition (like sqlalchemy does), the metaclass's |
| 145 | +# `__new__` and `__init__` are of no use to us. |
| 146 | +# |
| 147 | +# So a base class is really the right place to insert a custom `__new__` to |
| 148 | +# achieve what we want. |
| 149 | +class Singleton(metaclass=ThereCanBeOnlyOne): |
| 150 | + """Base class for singletons. Can be used as a mixin. |
| 151 | +
|
| 152 | + NOTE: Unpickling a singleton will retain the current instance, if it has already |
| 153 | + been created (in the current process). By default, its state is overwritten |
| 154 | + from the pickled data, by the default `__setstate__`. |
| 155 | + """ |
| 156 | + # We allow extra args so that __init__ can have them, but ignore them in the |
| 157 | + # super __new__ call, since our super is `object`, which takes no extra args. |
| 158 | + def __new__(cls, *args, **kwargs): |
| 159 | + if cls not in _instances: |
| 160 | + # Make a strong reference to keep the new instance alive until construction is done. |
| 161 | + instance = super().__new__(cls) |
| 162 | + _instances[cls] = instance |
| 163 | + return _instances[cls] |
| 164 | + |
| 165 | + |
| 166 | +# TODO: This won't work with classes that need another custom metaclass, |
| 167 | +# TODO: because then there's no unique most specific metaclass. |
| 168 | +# https://docs.python.org/3/reference/datamodel.html#determining-the-appropriate-metaclass |
| 169 | +# |
| 170 | +# **Workaround**: define a custom metaclass inheriting from all of those |
| 171 | +# metaclasses (including `ThereCanBeOnlyOne`). No body required; just "pass". |
| 172 | +# |
| 173 | +# (`ThereCanBeOnlyOne` is not officially part of the public API of this module, |
| 174 | +# but for this purpose, it's fine to use it directly. Hence no underscore in |
| 175 | +# the name, even though it's not listed in `__all__`.) |
| 176 | +# |
| 177 | +# Then use that as the metaclass for the class that both wants to be a singleton |
| 178 | +# and to use another metaclass for some other reason. The combined metaclass |
| 179 | +# will then satisfy the most-specific-metaclass constraint. |
| 180 | + |
| 181 | +# This is of course assuming that the metaclasses are orthogonal enough not to |
| 182 | +# interfere with each others' operation. If not, there is no general solution; |
| 183 | +# the specific situation must be sorted out by strategically overriding and |
| 184 | +# implementing any methods that conflict. |
| 185 | +# |
| 186 | +# |
| 187 | +# Proper solutions? |
| 188 | +# |
| 189 | +# - We can't drop our metaclass and enforce the don't-call-me-again |
| 190 | +# constraint in `Singleton.__new__`, because we need pickle to be able |
| 191 | +# to call `Singleton.__new__` while an instance already exists, to get |
| 192 | +# redirected to the existing instance. |
| 193 | +# |
| 194 | +# - The other option, providing a base class `__init__` to raise `TypeError` |
| 195 | +# at initialization time if an instance already exists, could also work, |
| 196 | +# because pickle skips `__init__`. |
| 197 | +# |
| 198 | +# However, this is not robust, as the derived class may easily forget to |
| 199 | +# call our `__init__`. In comparison, it's rare to customize `__new__`, |
| 200 | +# so that won't break as easily. |
| 201 | +# |
| 202 | +# Even assuming no mistakes in code that uses this, that requires more code |
| 203 | +# at each use site, for the super call; the whole point of this abstraction |
| 204 | +# being to condense the idea of "make this class a singleton", at the use |
| 205 | +# site, (beside the import) into just a single word. |
0 commit comments