1 :mod:`queue` --- A synchronized queue class

     2 ===========================================

     3 

     4 .. module:: Queue

     5    :synopsis: A synchronized queue class.

     6 

     7 .. note::

     8    The :mod:`Queue` module has been renamed to :mod:`queue` in Python 3.0.  The

     9    :term:`2to3` tool will automatically adapt imports when converting your

    10    sources to 3.0.

    11 

    12 

    13 The :mod:`Queue` module implements multi-producer, multi-consumer queues.

    14 It is especially useful in threaded programming when information must be

    15 exchanged safely between multiple threads.  The :class:`Queue` class in this

    16 module implements all the required locking semantics.  It depends on the

    17 availability of thread support in Python; see the :mod:`threading`

    18 module.

    19 

    20 Implements three types of queue whose only difference is the order that

    21 the entries are retrieved.  In a FIFO queue, the first tasks added are

    22 the first retrieved. In a LIFO queue, the most recently added entry is

    23 the first retrieved (operating like a stack).  With a priority queue,

    24 the entries are kept sorted (using the :mod:`heapq` module) and the

    25 lowest valued entry is retrieved first.

    26 

    27 The :mod:`Queue` module defines the following classes and exceptions:

    28 

    29 .. class:: Queue(maxsize)

    30 

    31    Constructor for a FIFO queue.  *maxsize* is an integer that sets the upperbound

    32    limit on the number of items that can be placed in the queue.  Insertion will

    33    block once this size has been reached, until queue items are consumed.  If

    34    *maxsize* is less than or equal to zero, the queue size is infinite.

    35 

    36 .. class:: LifoQueue(maxsize)

    37 

    38    Constructor for a LIFO queue.  *maxsize* is an integer that sets the upperbound

    39    limit on the number of items that can be placed in the queue.  Insertion will

    40    block once this size has been reached, until queue items are consumed.  If

    41    *maxsize* is less than or equal to zero, the queue size is infinite.

    42 

    43    .. versionadded:: 2.6

    44 

    45 .. class:: PriorityQueue(maxsize)

    46 

    47    Constructor for a priority queue.  *maxsize* is an integer that sets the upperbound

    48    limit on the number of items that can be placed in the queue.  Insertion will

    49    block once this size has been reached, until queue items are consumed.  If

    50    *maxsize* is less than or equal to zero, the queue size is infinite.

    51 

    52    The lowest valued entries are retrieved first (the lowest valued entry is the

    53    one returned by ``sorted(list(entries))[0]``).  A typical pattern for entries

    54    is a tuple in the form: ``(priority_number, data)``.

    55 

    56    .. versionadded:: 2.6

    57 

    58 .. exception:: Empty

    59 

    60    Exception raised when non-blocking :meth:`get` (or :meth:`get_nowait`) is called

    61    on a :class:`Queue` object which is empty.

    62 

    63 

    64 .. exception:: Full

    65 

    66    Exception raised when non-blocking :meth:`put` (or :meth:`put_nowait`) is called

    67    on a :class:`Queue` object which is full.

    68 

    69 

    70 .. _queueobjects:

    71 

    72 Queue Objects

    73 -------------

    74 

    75 Queue objects (:class:`Queue`, :class:`LifoQueue`, or :class:`PriorityQueue`)

    76 provide the public methods described below.

    77 

    78 

    79 .. method:: Queue.qsize()

    80 

    81    Return the approximate size of the queue.  Note, qsize() > 0 doesn't

    82    guarantee that a subsequent get() will not block, nor will qsize() < maxsize

    83    guarantee that put() will not block.

    84 

    85 

    86 .. method:: Queue.empty()

    87 

    88    Return ``True`` if the queue is empty, ``False`` otherwise.  If empty()

    89    returns ``True`` it doesn't guarantee that a subsequent call to put()

    90    will not block.  Similarly, if empty() returns ``False`` it doesn't

    91    guarantee that a subsequent call to get() will not block.

    92 

    93 

    94 .. method:: Queue.full()

    95 

    96    Return ``True`` if the queue is full, ``False`` otherwise.  If full()

    97    returns ``True`` it doesn't guarantee that a subsequent call to get()

    98    will not block.  Similarly, if full() returns ``False`` it doesn't

    99    guarantee that a subsequent call to put() will not block.

   100 

   101 

   102 .. method:: Queue.put(item[, block[, timeout]])

   103 

   104    Put *item* into the queue. If optional args *block* is true and *timeout* is

   105    None (the default), block if necessary until a free slot is available. If

   106    *timeout* is a positive number, it blocks at most *timeout* seconds and raises

   107    the :exc:`Full` exception if no free slot was available within that time.

   108    Otherwise (*block* is false), put an item on the queue if a free slot is

   109    immediately available, else raise the :exc:`Full` exception (*timeout* is

   110    ignored in that case).

   111 

   112    .. versionadded:: 2.3

   113       The *timeout* parameter.

   114 

   115 

   116 .. method:: Queue.put_nowait(item)

   117 

   118    Equivalent to ``put(item, False)``.

   119 

   120 

   121 .. method:: Queue.get([block[, timeout]])

   122 

   123    Remove and return an item from the queue. If optional args *block* is true and

   124    *timeout* is None (the default), block if necessary until an item is available.

   125    If *timeout* is a positive number, it blocks at most *timeout* seconds and

   126    raises the :exc:`Empty` exception if no item was available within that time.

   127    Otherwise (*block* is false), return an item if one is immediately available,

   128    else raise the :exc:`Empty` exception (*timeout* is ignored in that case).

   129 

   130    .. versionadded:: 2.3

   131       The *timeout* parameter.

   132 

   133 

   134 .. method:: Queue.get_nowait()

   135 

   136    Equivalent to ``get(False)``.

   137 

   138 Two methods are offered to support tracking whether enqueued tasks have been

   139 fully processed by daemon consumer threads.

   140 

   141 

   142 .. method:: Queue.task_done()

   143 

   144    Indicate that a formerly enqueued task is complete.  Used by queue consumer

   145    threads.  For each :meth:`get` used to fetch a task, a subsequent call to

   146    :meth:`task_done` tells the queue that the processing on the task is complete.

   147 

   148    If a :meth:`join` is currently blocking, it will resume when all items have been

   149    processed (meaning that a :meth:`task_done` call was received for every item

   150    that had been :meth:`put` into the queue).

   151 

   152    Raises a :exc:`ValueError` if called more times than there were items placed in

   153    the queue.

   154 

   155    .. versionadded:: 2.5

   156 

   157 

   158 .. method:: Queue.join()

   159 

   160    Blocks until all items in the queue have been gotten and processed.

   161 

   162    The count of unfinished tasks goes up whenever an item is added to the queue.

   163    The count goes down whenever a consumer thread calls :meth:`task_done` to

   164    indicate that the item was retrieved and all work on it is complete. When the

   165    count of unfinished tasks drops to zero, join() unblocks.

   166 

   167    .. versionadded:: 2.5

   168 

   169 Example of how to wait for enqueued tasks to be completed::

   170 

   171    def worker():

   172        while True:

   173            item = q.get()

   174            do_work(item)

   175            q.task_done()

   176 

   177    q = Queue()

   178    for i in range(num_worker_threads):

   179         t = Thread(target=worker)

   180         t.setDaemon(True)

   181         t.start()

   182 

   183    for item in source():

   184        q.put(item)

   185 

   186    q.join()       # block until all tasks are done

   187