|
1 |
|
2 :mod:`weakref` --- Weak references |
|
3 ================================== |
|
4 |
|
5 .. module:: weakref |
|
6 :synopsis: Support for weak references and weak dictionaries. |
|
7 .. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
|
8 .. moduleauthor:: Neil Schemenauer <nas@arctrix.com> |
|
9 .. moduleauthor:: Martin von Löwis <martin@loewis.home.cs.tu-berlin.de> |
|
10 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
|
11 |
|
12 |
|
13 .. versionadded:: 2.1 |
|
14 |
|
15 The :mod:`weakref` module allows the Python programmer to create :dfn:`weak |
|
16 references` to objects. |
|
17 |
|
18 .. When making changes to the examples in this file, be sure to update |
|
19 Lib/test/test_weakref.py::libreftest too! |
|
20 |
|
21 In the following, the term :dfn:`referent` means the object which is referred to |
|
22 by a weak reference. |
|
23 |
|
24 A weak reference to an object is not enough to keep the object alive: when the |
|
25 only remaining references to a referent are weak references, |
|
26 :term:`garbage collection` is free to destroy the referent and reuse its memory |
|
27 for something else. A primary use for weak references is to implement caches or |
|
28 mappings holding large objects, where it's desired that a large object not be |
|
29 kept alive solely because it appears in a cache or mapping. |
|
30 |
|
31 For example, if you have a number of large binary image objects, you may wish to |
|
32 associate a name with each. If you used a Python dictionary to map names to |
|
33 images, or images to names, the image objects would remain alive just because |
|
34 they appeared as values or keys in the dictionaries. The |
|
35 :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by |
|
36 the :mod:`weakref` module are an alternative, using weak references to construct |
|
37 mappings that don't keep objects alive solely because they appear in the mapping |
|
38 objects. If, for example, an image object is a value in a |
|
39 :class:`WeakValueDictionary`, then when the last remaining references to that |
|
40 image object are the weak references held by weak mappings, garbage collection |
|
41 can reclaim the object, and its corresponding entries in weak mappings are |
|
42 simply deleted. |
|
43 |
|
44 :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references |
|
45 in their implementation, setting up callback functions on the weak references |
|
46 that notify the weak dictionaries when a key or value has been reclaimed by |
|
47 garbage collection. Most programs should find that using one of these weak |
|
48 dictionary types is all they need -- it's not usually necessary to create your |
|
49 own weak references directly. The low-level machinery used by the weak |
|
50 dictionary implementations is exposed by the :mod:`weakref` module for the |
|
51 benefit of advanced uses. |
|
52 |
|
53 .. note:: |
|
54 |
|
55 Weak references to an object are cleared before the object's :meth:`__del__` |
|
56 is called, to ensure that the weak reference callback (if any) finds the |
|
57 object still alive. |
|
58 |
|
59 Not all objects can be weakly referenced; those objects which can include class |
|
60 instances, functions written in Python (but not in C), methods (both bound and |
|
61 unbound), sets, frozensets, file objects, :term:`generator`\s, type objects, |
|
62 :class:`DBcursor` objects from the :mod:`bsddb` module, sockets, arrays, deques, |
|
63 and regular expression pattern objects. |
|
64 |
|
65 .. versionchanged:: 2.4 |
|
66 Added support for files, sockets, arrays, and patterns. |
|
67 |
|
68 Several builtin types such as :class:`list` and :class:`dict` do not directly |
|
69 support weak references but can add support through subclassing:: |
|
70 |
|
71 class Dict(dict): |
|
72 pass |
|
73 |
|
74 obj = Dict(red=1, green=2, blue=3) # this object is weak referenceable |
|
75 |
|
76 Extension types can easily be made to support weak references; see |
|
77 :ref:`weakref-support`. |
|
78 |
|
79 |
|
80 .. class:: ref(object[, callback]) |
|
81 |
|
82 Return a weak reference to *object*. The original object can be retrieved by |
|
83 calling the reference object if the referent is still alive; if the referent is |
|
84 no longer alive, calling the reference object will cause :const:`None` to be |
|
85 returned. If *callback* is provided and not :const:`None`, and the returned |
|
86 weakref object is still alive, the callback will be called when the object is |
|
87 about to be finalized; the weak reference object will be passed as the only |
|
88 parameter to the callback; the referent will no longer be available. |
|
89 |
|
90 It is allowable for many weak references to be constructed for the same object. |
|
91 Callbacks registered for each weak reference will be called from the most |
|
92 recently registered callback to the oldest registered callback. |
|
93 |
|
94 Exceptions raised by the callback will be noted on the standard error output, |
|
95 but cannot be propagated; they are handled in exactly the same way as exceptions |
|
96 raised from an object's :meth:`__del__` method. |
|
97 |
|
98 Weak references are :term:`hashable` if the *object* is hashable. They will maintain |
|
99 their hash value even after the *object* was deleted. If :func:`hash` is called |
|
100 the first time only after the *object* was deleted, the call will raise |
|
101 :exc:`TypeError`. |
|
102 |
|
103 Weak references support tests for equality, but not ordering. If the referents |
|
104 are still alive, two references have the same equality relationship as their |
|
105 referents (regardless of the *callback*). If either referent has been deleted, |
|
106 the references are equal only if the reference objects are the same object. |
|
107 |
|
108 .. versionchanged:: 2.4 |
|
109 This is now a subclassable type rather than a factory function; it derives from |
|
110 :class:`object`. |
|
111 |
|
112 |
|
113 .. function:: proxy(object[, callback]) |
|
114 |
|
115 Return a proxy to *object* which uses a weak reference. This supports use of |
|
116 the proxy in most contexts instead of requiring the explicit dereferencing used |
|
117 with weak reference objects. The returned object will have a type of either |
|
118 ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is |
|
119 callable. Proxy objects are not :term:`hashable` regardless of the referent; this |
|
120 avoids a number of problems related to their fundamentally mutable nature, and |
|
121 prevent their use as dictionary keys. *callback* is the same as the parameter |
|
122 of the same name to the :func:`ref` function. |
|
123 |
|
124 |
|
125 .. function:: getweakrefcount(object) |
|
126 |
|
127 Return the number of weak references and proxies which refer to *object*. |
|
128 |
|
129 |
|
130 .. function:: getweakrefs(object) |
|
131 |
|
132 Return a list of all weak reference and proxy objects which refer to *object*. |
|
133 |
|
134 |
|
135 .. class:: WeakKeyDictionary([dict]) |
|
136 |
|
137 Mapping class that references keys weakly. Entries in the dictionary will be |
|
138 discarded when there is no longer a strong reference to the key. This can be |
|
139 used to associate additional data with an object owned by other parts of an |
|
140 application without adding attributes to those objects. This can be especially |
|
141 useful with objects that override attribute accesses. |
|
142 |
|
143 .. note:: |
|
144 |
|
145 Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python |
|
146 dictionary, it must not change size when iterating over it. This can be |
|
147 difficult to ensure for a :class:`WeakKeyDictionary` because actions |
|
148 performed by the program during iteration may cause items in the |
|
149 dictionary to vanish "by magic" (as a side effect of garbage collection). |
|
150 |
|
151 :class:`WeakKeyDictionary` objects have the following additional methods. These |
|
152 expose the internal references directly. The references are not guaranteed to |
|
153 be "live" at the time they are used, so the result of calling the references |
|
154 needs to be checked before being used. This can be used to avoid creating |
|
155 references that will cause the garbage collector to keep the keys around longer |
|
156 than needed. |
|
157 |
|
158 |
|
159 .. method:: WeakKeyDictionary.iterkeyrefs() |
|
160 |
|
161 Return an :term:`iterator` that yields the weak references to the keys. |
|
162 |
|
163 .. versionadded:: 2.5 |
|
164 |
|
165 |
|
166 .. method:: WeakKeyDictionary.keyrefs() |
|
167 |
|
168 Return a list of weak references to the keys. |
|
169 |
|
170 .. versionadded:: 2.5 |
|
171 |
|
172 |
|
173 .. class:: WeakValueDictionary([dict]) |
|
174 |
|
175 Mapping class that references values weakly. Entries in the dictionary will be |
|
176 discarded when no strong reference to the value exists any more. |
|
177 |
|
178 .. note:: |
|
179 |
|
180 Caution: Because a :class:`WeakValueDictionary` is built on top of a Python |
|
181 dictionary, it must not change size when iterating over it. This can be |
|
182 difficult to ensure for a :class:`WeakValueDictionary` because actions performed |
|
183 by the program during iteration may cause items in the dictionary to vanish "by |
|
184 magic" (as a side effect of garbage collection). |
|
185 |
|
186 :class:`WeakValueDictionary` objects have the following additional methods. |
|
187 These method have the same issues as the :meth:`iterkeyrefs` and :meth:`keyrefs` |
|
188 methods of :class:`WeakKeyDictionary` objects. |
|
189 |
|
190 |
|
191 .. method:: WeakValueDictionary.itervaluerefs() |
|
192 |
|
193 Return an :term:`iterator` that yields the weak references to the values. |
|
194 |
|
195 .. versionadded:: 2.5 |
|
196 |
|
197 |
|
198 .. method:: WeakValueDictionary.valuerefs() |
|
199 |
|
200 Return a list of weak references to the values. |
|
201 |
|
202 .. versionadded:: 2.5 |
|
203 |
|
204 |
|
205 .. data:: ReferenceType |
|
206 |
|
207 The type object for weak references objects. |
|
208 |
|
209 |
|
210 .. data:: ProxyType |
|
211 |
|
212 The type object for proxies of objects which are not callable. |
|
213 |
|
214 |
|
215 .. data:: CallableProxyType |
|
216 |
|
217 The type object for proxies of callable objects. |
|
218 |
|
219 |
|
220 .. data:: ProxyTypes |
|
221 |
|
222 Sequence containing all the type objects for proxies. This can make it simpler |
|
223 to test if an object is a proxy without being dependent on naming both proxy |
|
224 types. |
|
225 |
|
226 |
|
227 .. exception:: ReferenceError |
|
228 |
|
229 Exception raised when a proxy object is used but the underlying object has been |
|
230 collected. This is the same as the standard :exc:`ReferenceError` exception. |
|
231 |
|
232 |
|
233 .. seealso:: |
|
234 |
|
235 :pep:`0205` - Weak References |
|
236 The proposal and rationale for this feature, including links to earlier |
|
237 implementations and information about similar features in other languages. |
|
238 |
|
239 |
|
240 .. _weakref-objects: |
|
241 |
|
242 Weak Reference Objects |
|
243 ---------------------- |
|
244 |
|
245 Weak reference objects have no attributes or methods, but do allow the referent |
|
246 to be obtained, if it still exists, by calling it: |
|
247 |
|
248 >>> import weakref |
|
249 >>> class Object: |
|
250 ... pass |
|
251 ... |
|
252 >>> o = Object() |
|
253 >>> r = weakref.ref(o) |
|
254 >>> o2 = r() |
|
255 >>> o is o2 |
|
256 True |
|
257 |
|
258 If the referent no longer exists, calling the reference object returns |
|
259 :const:`None`: |
|
260 |
|
261 >>> del o, o2 |
|
262 >>> print r() |
|
263 None |
|
264 |
|
265 Testing that a weak reference object is still live should be done using the |
|
266 expression ``ref() is not None``. Normally, application code that needs to use |
|
267 a reference object should follow this pattern:: |
|
268 |
|
269 # r is a weak reference object |
|
270 o = r() |
|
271 if o is None: |
|
272 # referent has been garbage collected |
|
273 print "Object has been deallocated; can't frobnicate." |
|
274 else: |
|
275 print "Object is still live!" |
|
276 o.do_something_useful() |
|
277 |
|
278 Using a separate test for "liveness" creates race conditions in threaded |
|
279 applications; another thread can cause a weak reference to become invalidated |
|
280 before the weak reference is called; the idiom shown above is safe in threaded |
|
281 applications as well as single-threaded applications. |
|
282 |
|
283 Specialized versions of :class:`ref` objects can be created through subclassing. |
|
284 This is used in the implementation of the :class:`WeakValueDictionary` to reduce |
|
285 the memory overhead for each entry in the mapping. This may be most useful to |
|
286 associate additional information with a reference, but could also be used to |
|
287 insert additional processing on calls to retrieve the referent. |
|
288 |
|
289 This example shows how a subclass of :class:`ref` can be used to store |
|
290 additional information about an object and affect the value that's returned when |
|
291 the referent is accessed:: |
|
292 |
|
293 import weakref |
|
294 |
|
295 class ExtendedRef(weakref.ref): |
|
296 def __init__(self, ob, callback=None, **annotations): |
|
297 super(ExtendedRef, self).__init__(ob, callback) |
|
298 self.__counter = 0 |
|
299 for k, v in annotations.iteritems(): |
|
300 setattr(self, k, v) |
|
301 |
|
302 def __call__(self): |
|
303 """Return a pair containing the referent and the number of |
|
304 times the reference has been called. |
|
305 """ |
|
306 ob = super(ExtendedRef, self).__call__() |
|
307 if ob is not None: |
|
308 self.__counter += 1 |
|
309 ob = (ob, self.__counter) |
|
310 return ob |
|
311 |
|
312 |
|
313 .. _weakref-example: |
|
314 |
|
315 Example |
|
316 ------- |
|
317 |
|
318 This simple example shows how an application can use objects IDs to retrieve |
|
319 objects that it has seen before. The IDs of the objects can then be used in |
|
320 other data structures without forcing the objects to remain alive, but the |
|
321 objects can still be retrieved by ID if they do. |
|
322 |
|
323 .. Example contributed by Tim Peters. |
|
324 |
|
325 :: |
|
326 |
|
327 import weakref |
|
328 |
|
329 _id2obj_dict = weakref.WeakValueDictionary() |
|
330 |
|
331 def remember(obj): |
|
332 oid = id(obj) |
|
333 _id2obj_dict[oid] = obj |
|
334 return oid |
|
335 |
|
336 def id2obj(oid): |
|
337 return _id2obj_dict[oid] |
|
338 |