|
1 |
|
2 :mod:`shelve` --- Python object persistence |
|
3 =========================================== |
|
4 |
|
5 .. module:: shelve |
|
6 :synopsis: Python object persistence. |
|
7 |
|
8 |
|
9 .. index:: module: pickle |
|
10 |
|
11 A "shelf" is a persistent, dictionary-like object. The difference with "dbm" |
|
12 databases is that the values (not the keys!) in a shelf can be essentially |
|
13 arbitrary Python objects --- anything that the :mod:`pickle` module can handle. |
|
14 This includes most class instances, recursive data types, and objects containing |
|
15 lots of shared sub-objects. The keys are ordinary strings. |
|
16 |
|
17 |
|
18 .. function:: open(filename[, flag='c'[, protocol=None[, writeback=False]]]) |
|
19 |
|
20 Open a persistent dictionary. The filename specified is the base filename for |
|
21 the underlying database. As a side-effect, an extension may be added to the |
|
22 filename and more than one file may be created. By default, the underlying |
|
23 database file is opened for reading and writing. The optional *flag* parameter |
|
24 has the same interpretation as the *flag* parameter of :func:`anydbm.open`. |
|
25 |
|
26 By default, version 0 pickles are used to serialize values. The version of the |
|
27 pickle protocol can be specified with the *protocol* parameter. |
|
28 |
|
29 .. versionchanged:: 2.3 |
|
30 The *protocol* parameter was added. |
|
31 |
|
32 By default, mutations to persistent-dictionary mutable entries are not |
|
33 automatically written back. If the optional *writeback* parameter is set to |
|
34 *True*, all entries accessed are cached in memory, and written back at close |
|
35 time; this can make it handier to mutate mutable entries in the persistent |
|
36 dictionary, but, if many entries are accessed, it can consume vast amounts of |
|
37 memory for the cache, and it can make the close operation very slow since all |
|
38 accessed entries are written back (there is no way to determine which accessed |
|
39 entries are mutable, nor which ones were actually mutated). |
|
40 |
|
41 Shelve objects support all methods supported by dictionaries. This eases the |
|
42 transition from dictionary based scripts to those requiring persistent storage. |
|
43 |
|
44 One additional method is supported: |
|
45 |
|
46 |
|
47 .. method:: Shelf.sync() |
|
48 |
|
49 Write back all entries in the cache if the shelf was opened with *writeback* set |
|
50 to *True*. Also empty the cache and synchronize the persistent dictionary on |
|
51 disk, if feasible. This is called automatically when the shelf is closed with |
|
52 :meth:`close`. |
|
53 |
|
54 |
|
55 Restrictions |
|
56 ------------ |
|
57 |
|
58 .. index:: |
|
59 module: dbm |
|
60 module: gdbm |
|
61 module: bsddb |
|
62 |
|
63 * The choice of which database package will be used (such as :mod:`dbm`, |
|
64 :mod:`gdbm` or :mod:`bsddb`) depends on which interface is available. Therefore |
|
65 it is not safe to open the database directly using :mod:`dbm`. The database is |
|
66 also (unfortunately) subject to the limitations of :mod:`dbm`, if it is used --- |
|
67 this means that (the pickled representation of) the objects stored in the |
|
68 database should be fairly small, and in rare cases key collisions may cause the |
|
69 database to refuse updates. |
|
70 |
|
71 * Depending on the implementation, closing a persistent dictionary may or may |
|
72 not be necessary to flush changes to disk. The :meth:`__del__` method of the |
|
73 :class:`Shelf` class calls the :meth:`close` method, so the programmer generally |
|
74 need not do this explicitly. |
|
75 |
|
76 * The :mod:`shelve` module does not support *concurrent* read/write access to |
|
77 shelved objects. (Multiple simultaneous read accesses are safe.) When a |
|
78 program has a shelf open for writing, no other program should have it open for |
|
79 reading or writing. Unix file locking can be used to solve this, but this |
|
80 differs across Unix versions and requires knowledge about the database |
|
81 implementation used. |
|
82 |
|
83 |
|
84 .. class:: Shelf(dict[, protocol=None[, writeback=False]]) |
|
85 |
|
86 A subclass of :class:`UserDict.DictMixin` which stores pickled values in the |
|
87 *dict* object. |
|
88 |
|
89 By default, version 0 pickles are used to serialize values. The version of the |
|
90 pickle protocol can be specified with the *protocol* parameter. See the |
|
91 :mod:`pickle` documentation for a discussion of the pickle protocols. |
|
92 |
|
93 .. versionchanged:: 2.3 |
|
94 The *protocol* parameter was added. |
|
95 |
|
96 If the *writeback* parameter is ``True``, the object will hold a cache of all |
|
97 entries accessed and write them back to the *dict* at sync and close times. |
|
98 This allows natural operations on mutable entries, but can consume much more |
|
99 memory and make sync and close take a long time. |
|
100 |
|
101 |
|
102 .. class:: BsdDbShelf(dict[, protocol=None[, writeback=False]]) |
|
103 |
|
104 A subclass of :class:`Shelf` which exposes :meth:`first`, :meth:`next`, |
|
105 :meth:`previous`, :meth:`last` and :meth:`set_location` which are available in |
|
106 the :mod:`bsddb` module but not in other database modules. The *dict* object |
|
107 passed to the constructor must support those methods. This is generally |
|
108 accomplished by calling one of :func:`bsddb.hashopen`, :func:`bsddb.btopen` or |
|
109 :func:`bsddb.rnopen`. The optional *protocol* and *writeback* parameters have |
|
110 the same interpretation as for the :class:`Shelf` class. |
|
111 |
|
112 |
|
113 .. class:: DbfilenameShelf(filename[, flag='c'[, protocol=None[, writeback=False]]]) |
|
114 |
|
115 A subclass of :class:`Shelf` which accepts a *filename* instead of a dict-like |
|
116 object. The underlying file will be opened using :func:`anydbm.open`. By |
|
117 default, the file will be created and opened for both read and write. The |
|
118 optional *flag* parameter has the same interpretation as for the :func:`open` |
|
119 function. The optional *protocol* and *writeback* parameters have the same |
|
120 interpretation as for the :class:`Shelf` class. |
|
121 |
|
122 |
|
123 Example |
|
124 ------- |
|
125 |
|
126 To summarize the interface (``key`` is a string, ``data`` is an arbitrary |
|
127 object):: |
|
128 |
|
129 import shelve |
|
130 |
|
131 d = shelve.open(filename) # open -- file may get suffix added by low-level |
|
132 # library |
|
133 |
|
134 d[key] = data # store data at key (overwrites old data if |
|
135 # using an existing key) |
|
136 data = d[key] # retrieve a COPY of data at key (raise KeyError if no |
|
137 # such key) |
|
138 del d[key] # delete data stored at key (raises KeyError |
|
139 # if no such key) |
|
140 flag = d.has_key(key) # true if the key exists |
|
141 klist = d.keys() # a list of all existing keys (slow!) |
|
142 |
|
143 # as d was opened WITHOUT writeback=True, beware: |
|
144 d['xx'] = range(4) # this works as expected, but... |
|
145 d['xx'].append(5) # *this doesn't!* -- d['xx'] is STILL range(4)!!! |
|
146 |
|
147 # having opened d without writeback=True, you need to code carefully: |
|
148 temp = d['xx'] # extracts the copy |
|
149 temp.append(5) # mutates the copy |
|
150 d['xx'] = temp # stores the copy right back, to persist it |
|
151 |
|
152 # or, d=shelve.open(filename,writeback=True) would let you just code |
|
153 # d['xx'].append(5) and have it work as expected, BUT it would also |
|
154 # consume more memory and make the d.close() operation slower. |
|
155 |
|
156 d.close() # close it |
|
157 |
|
158 |
|
159 .. seealso:: |
|
160 |
|
161 Module :mod:`anydbm` |
|
162 Generic interface to ``dbm``\ -style databases. |
|
163 |
|
164 Module :mod:`bsddb` |
|
165 BSD ``db`` database interface. |
|
166 |
|
167 Module :mod:`dbhash` |
|
168 Thin layer around the :mod:`bsddb` which provides an :func:`open` function like |
|
169 the other database modules. |
|
170 |
|
171 Module :mod:`dbm` |
|
172 Standard Unix database interface. |
|
173 |
|
174 Module :mod:`dumbdbm` |
|
175 Portable implementation of the ``dbm`` interface. |
|
176 |
|
177 Module :mod:`gdbm` |
|
178 GNU database interface, based on the ``dbm`` interface. |
|
179 |
|
180 Module :mod:`pickle` |
|
181 Object serialization used by :mod:`shelve`. |
|
182 |
|
183 Module :mod:`cPickle` |
|
184 High-performance version of :mod:`pickle`. |
|
185 |