1 

     2 :mod:`mmap` --- Memory-mapped file support

     3 ==========================================

     4 

     5 .. module:: mmap

     6    :synopsis: Interface to memory-mapped files for Unix and Windows.

     7 

     8 

     9 Memory-mapped file objects behave like both strings and like file objects.

    10 Unlike normal string objects, however, these are mutable.  You can use mmap

    11 objects in most places where strings are expected; for example, you can use

    12 the :mod:`re` module to search through a memory-mapped file.  Since they're

    13 mutable, you can change a single character by doing ``obj[index] = 'a'``, or

    14 change a substring by assigning to a slice: ``obj[i1:i2] = '...'``.  You can

    15 also read and write data starting at the current file position, and

    16 :meth:`seek` through the file to different positions.

    17 

    18 A memory-mapped file is created by the :class:`mmap` constructor, which is

    19 different on Unix and on Windows.  In either case you must provide a file

    20 descriptor for a file opened for update. If you wish to map an existing Python

    21 file object, use its :meth:`fileno` method to obtain the correct value for the

    22 *fileno* parameter.  Otherwise, you can open the file using the

    23 :func:`os.open` function, which returns a file descriptor directly (the file

    24 still needs to be closed when done).

    25 

    26 For both the Unix and Windows versions of the constructor, *access* may be

    27 specified as an optional keyword parameter. *access* accepts one of three

    28 values: :const:`ACCESS_READ`, :const:`ACCESS_WRITE`, or :const:`ACCESS_COPY`

    29 to specify read-only, write-through or copy-on-write memory respectively.

    30 *access* can be used on both Unix and Windows.  If *access* is not specified,

    31 Windows mmap returns a write-through mapping.  The initial memory values for

    32 all three access types are taken from the specified file.  Assignment to an

    33 :const:`ACCESS_READ` memory map raises a :exc:`TypeError` exception.

    34 Assignment to an :const:`ACCESS_WRITE` memory map affects both memory and the

    35 underlying file.  Assignment to an :const:`ACCESS_COPY` memory map affects

    36 memory but does not update the underlying file.

    37 

    38 .. versionchanged:: 2.5

    39    To map anonymous memory, -1 should be passed as the fileno along with the

    40    length.

    41 

    42 .. versionchanged:: 2.6

    43    mmap.mmap has formerly been a factory function creating mmap objects. Now

    44    mmap.mmap is the class itself.

    45 

    46 .. class:: mmap(fileno, length[, tagname[, access[, offset]]])

    47 

    48    **(Windows version)** Maps *length* bytes from the file specified by the

    49    file handle *fileno*, and creates a mmap object.  If *length* is larger

    50    than the current size of the file, the file is extended to contain *length*

    51    bytes.  If *length* is ``0``, the maximum length of the map is the current

    52    size of the file, except that if the file is empty Windows raises an

    53    exception (you cannot create an empty mapping on Windows).

    54 

    55    *tagname*, if specified and not ``None``, is a string giving a tag name for

    56    the mapping.  Windows allows you to have many different mappings against

    57    the same file.  If you specify the name of an existing tag, that tag is

    58    opened, otherwise a new tag of this name is created.  If this parameter is

    59    omitted or ``None``, the mapping is created without a name.  Avoiding the

    60    use of the tag parameter will assist in keeping your code portable between

    61    Unix and Windows.

    62 

    63    *offset* may be specified as a non-negative integer offset. mmap references

    64    will be relative to the offset from the beginning of the file. *offset*

    65    defaults to 0.  *offset* must be a multiple of the ALLOCATIONGRANULARITY.

    66 

    67 

    68 .. class:: mmap(fileno, length[, flags[, prot[, access[, offset]]]])

    69    :noindex:

    70 

    71    **(Unix version)** Maps *length* bytes from the file specified by the file

    72    descriptor *fileno*, and returns a mmap object.  If *length* is ``0``, the

    73    maximum length of the map will be the current size of the file when

    74    :class:`mmap` is called.

    75 

    76    *flags* specifies the nature of the mapping. :const:`MAP_PRIVATE` creates a

    77    private copy-on-write mapping, so changes to the contents of the mmap

    78    object will be private to this process, and :const:`MAP_SHARED` creates a

    79    mapping that's shared with all other processes mapping the same areas of

    80    the file.  The default value is :const:`MAP_SHARED`.

    81 

    82    *prot*, if specified, gives the desired memory protection; the two most

    83    useful values are :const:`PROT_READ` and :const:`PROT_WRITE`, to specify

    84    that the pages may be read or written.  *prot* defaults to

    85    :const:`PROT_READ \| PROT_WRITE`.

    86 

    87    *access* may be specified in lieu of *flags* and *prot* as an optional

    88    keyword parameter.  It is an error to specify both *flags*, *prot* and

    89    *access*.  See the description of *access* above for information on how to

    90    use this parameter.

    91 

    92    *offset* may be specified as a non-negative integer offset. mmap references

    93    will be relative to the offset from the beginning of the file. *offset*

    94    defaults to 0.  *offset* must be a multiple of the PAGESIZE or

    95    ALLOCATIONGRANULARITY.

    96    

    97    This example shows a simple way of using :class:`mmap`::

    98 

    99       import mmap

   100 

   101       # write a simple example file

   102       with open("hello.txt", "w") as f:

   103           f.write("Hello Python!\n")

   104 

   105       with open("hello.txt", "r+") as f:

   106           # memory-map the file, size 0 means whole file

   107           map = mmap.mmap(f.fileno(), 0)

   108           # read content via standard file methods

   109           print map.readline()  # prints "Hello Python!"

   110           # read content via slice notation

   111           print map[:5]  # prints "Hello"

   112           # update content using slice notation;

   113           # note that new content must have same size

   114           map[6:] = " world!\n"

   115           # ... and read again using standard file methods

   116           map.seek(0)

   117           print map.readline()  # prints "Hello  world!"

   118           # close the map

   119           map.close()

   120 

   121 

   122    The next example demonstrates how to create an anonymous map and exchange

   123    data between the parent and child processes::

   124 

   125       import mmap

   126       import os

   127 

   128       map = mmap.mmap(-1, 13)

   129       map.write("Hello world!")

   130 

   131       pid = os.fork()

   132 

   133       if pid == 0: # In a child process

   134           map.seek(0)

   135           print map.readline()

   136 

   137           map.close()

   138 

   139 

   140    Memory-mapped file objects support the following methods:

   141 

   142 

   143    .. method:: close()

   144 

   145       Close the file.  Subsequent calls to other methods of the object will

   146       result in an exception being raised.

   147 

   148 

   149    .. method:: find(string[, start[, end]])

   150 

   151       Returns the lowest index in the object where the substring *string* is

   152       found, such that *string* is contained in the range [*start*, *end*].

   153       Optional arguments *start* and *end* are interpreted as in slice notation.

   154       Returns ``-1`` on failure.

   155 

   156 

   157    .. method:: flush([offset, size])

   158 

   159       Flushes changes made to the in-memory copy of a file back to disk. Without

   160       use of this call there is no guarantee that changes are written back before

   161       the object is destroyed.  If *offset* and *size* are specified, only

   162       changes to the given range of bytes will be flushed to disk; otherwise, the

   163       whole extent of the mapping is flushed.

   164 

   165       **(Windows version)** A nonzero value returned indicates success; zero

   166       indicates failure.

   167 

   168       **(Unix version)** A zero value is returned to indicate success. An

   169       exception is raised when the call failed.

   170 

   171 

   172    .. method:: move(dest, src, count)

   173 

   174       Copy the *count* bytes starting at offset *src* to the destination index

   175       *dest*.  If the mmap was created with :const:`ACCESS_READ`, then calls to

   176       move will throw a :exc:`TypeError` exception.

   177 

   178 

   179    .. method:: read(num)

   180 

   181       Return a string containing up to *num* bytes starting from the current

   182       file position; the file position is updated to point after the bytes that

   183       were returned.

   184 

   185 

   186    .. method:: read_byte()

   187 

   188       Returns a string of length 1 containing the character at the current file

   189       position, and advances the file position by 1.

   190 

   191 

   192    .. method:: readline()

   193 

   194       Returns a single line, starting at the current file position and up to the

   195       next newline.

   196 

   197 

   198    .. method:: resize(newsize)

   199 

   200       Resizes the map and the underlying file, if any. If the mmap was created

   201       with :const:`ACCESS_READ` or :const:`ACCESS_COPY`, resizing the map will

   202       throw a :exc:`TypeError` exception.

   203 

   204 

   205    .. method:: rfind(string[, start[, end]])

   206 

   207       Returns the highest index in the object where the substring *string* is

   208       found, such that *string* is contained in the range [*start*, *end*].

   209       Optional arguments *start* and *end* are interpreted as in slice notation.

   210       Returns ``-1`` on failure.

   211 

   212 

   213    .. method:: seek(pos[, whence])

   214 

   215       Set the file's current position.  *whence* argument is optional and

   216       defaults to ``os.SEEK_SET`` or ``0`` (absolute file positioning); other

   217       values are ``os.SEEK_CUR`` or ``1`` (seek relative to the current

   218       position) and ``os.SEEK_END`` or ``2`` (seek relative to the file's end).

   219 

   220 

   221    .. method:: size()

   222 

   223       Return the length of the file, which can be larger than the size of the

   224       memory-mapped area.

   225 

   226 

   227    .. method:: tell()

   228 

   229       Returns the current position of the file pointer.

   230 

   231 

   232    .. method:: write(string)

   233 

   234       Write the bytes in *string* into memory at the current position of the

   235       file pointer; the file position is updated to point after the bytes that

   236       were written. If the mmap was created with :const:`ACCESS_READ`, then

   237       writing to it will throw a :exc:`TypeError` exception.

   238 

   239 

   240    .. method:: write_byte(byte)

   241 

   242       Write the single-character string *byte* into memory at the current

   243       position of the file pointer; the file position is advanced by ``1``. If

   244       the mmap was created with :const:`ACCESS_READ`, then writing to it will

   245       throw a :exc:`TypeError` exception.

   246 

   247