1 

     2 :mod:`asynchat` --- Asynchronous socket command/response handler

     3 ================================================================

     4 

     5 .. module:: asynchat

     6    :synopsis: Support for asynchronous command/response protocols.

     7 .. moduleauthor:: Sam Rushing <rushing@nightmare.com>

     8 .. sectionauthor:: Steve Holden <sholden@holdenweb.com>

     9 

    10 

    11 This module builds on the :mod:`asyncore` infrastructure, simplifying

    12 asynchronous clients and servers and making it easier to handle protocols

    13 whose elements are terminated by arbitrary strings, or are of variable length.

    14 :mod:`asynchat` defines the abstract class :class:`async_chat` that you

    15 subclass, providing implementations of the :meth:`collect_incoming_data` and

    16 :meth:`found_terminator` methods. It uses the same asynchronous loop as

    17 :mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher`

    18 and :class:`asynchat.async_chat`, can freely be mixed in the channel map.

    19 Typically an :class:`asyncore.dispatcher` server channel generates new

    20 :class:`asynchat.async_chat` channel objects as it receives incoming

    21 connection requests.

    22 

    23 

    24 .. class:: async_chat()

    25 

    26    This class is an abstract subclass of :class:`asyncore.dispatcher`. To make

    27    practical use of the code you must subclass :class:`async_chat`, providing

    28    meaningful :meth:`collect_incoming_data` and :meth:`found_terminator`

    29    methods.

    30    The :class:`asyncore.dispatcher` methods can be used, although not all make

    31    sense in a message/response context.

    32 

    33    Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of

    34    events that are generated by an analysis of socket conditions after a

    35    :cfunc:`select` call. Once the polling loop has been started the

    36    :class:`async_chat` object's methods are called by the event-processing

    37    framework with no action on the part of the programmer.

    38 

    39    Two class attributes can be modified, to improve performance, or possibly

    40    even to conserve memory.

    41 

    42 

    43    .. data:: ac_in_buffer_size

    44 

    45       The asynchronous input buffer size (default ``4096``).

    46 

    47 

    48    .. data:: ac_out_buffer_size

    49 

    50       The asynchronous output buffer size (default ``4096``).

    51 

    52    Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to

    53    define a first-in-first-out queue (fifo) of *producers*. A producer need

    54    have only one method, :meth:`more`, which should return data to be

    55    transmitted on the channel.

    56    The producer indicates exhaustion (*i.e.* that it contains no more data) by

    57    having its :meth:`more` method return the empty string. At this point the

    58    :class:`async_chat` object removes the producer from the fifo and starts

    59    using the next producer, if any. When the producer fifo is empty the

    60    :meth:`handle_write` method does nothing. You use the channel object's

    61    :meth:`set_terminator` method to describe how to recognize the end of, or

    62    an important breakpoint in, an incoming transmission from the remote

    63    endpoint.

    64 

    65    To build a functioning :class:`async_chat` subclass your  input methods

    66    :meth:`collect_incoming_data` and :meth:`found_terminator` must handle the

    67    data that the channel receives asynchronously. The methods are described

    68    below.

    69 

    70 

    71 .. method:: async_chat.close_when_done()

    72 

    73    Pushes a ``None`` on to the producer fifo. When this producer is popped off

    74    the fifo it causes the channel to be closed.

    75 

    76 

    77 .. method:: async_chat.collect_incoming_data(data)

    78 

    79    Called with *data* holding an arbitrary amount of received data.  The

    80    default method, which must be overridden, raises a

    81    :exc:`NotImplementedError` exception.

    82 

    83 

    84 .. method:: async_chat._collect_incoming_data(data)

    85 

    86    Sample implementation of a data collection rutine to be used in conjunction

    87    with :meth:`_get_data` in a user-specified :meth:`found_terminator`.

    88 

    89 

    90 .. method:: async_chat.discard_buffers()

    91 

    92    In emergencies this method will discard any data held in the input and/or

    93    output buffers and the producer fifo.

    94 

    95 

    96 .. method:: async_chat.found_terminator()

    97 

    98    Called when the incoming data stream  matches the termination condition set

    99    by :meth:`set_terminator`. The default method, which must be overridden,

   100    raises a :exc:`NotImplementedError` exception. The buffered input data

   101    should be available via an instance attribute.

   102 

   103 

   104 .. method:: async_chat._get_data()

   105 

   106    Will return and clear the data received with the sample

   107    :meth:`_collect_incoming_data` implementation.

   108 

   109 

   110 .. method:: async_chat.get_terminator()

   111 

   112    Returns the current terminator for the channel.

   113 

   114 

   115 .. method:: async_chat.handle_close()

   116 

   117    Called when the channel is closed. The default method silently closes the

   118    channel's socket.

   119 

   120 

   121 .. method:: async_chat.handle_read()

   122 

   123    Called when a read event fires on the channel's socket in the asynchronous

   124    loop.  The default method checks for the termination condition established

   125    by :meth:`set_terminator`, which can be either the appearance of a

   126    particular string in the input stream or the receipt of a particular number

   127    of characters.  When the terminator is found, :meth:`handle_read` calls the

   128    :meth:`found_terminator` method after calling :meth:`collect_incoming_data`

   129    with any data preceding the terminating condition.

   130 

   131 

   132 .. method:: async_chat.handle_write()

   133 

   134    Called when the application may write data to the channel.   The default

   135    method calls the :meth:`initiate_send` method, which in turn will call

   136    :meth:`refill_buffer` to collect data from the producer fifo associated

   137    with the channel.

   138 

   139 

   140 .. method:: async_chat.push(data)

   141 

   142    Creates a :class:`simple_producer` object (*see below*) containing the data

   143    and pushes it on to the channel's ``producer_fifo`` to ensure its

   144    transmission.  This is all you need to do to have the channel write the

   145    data out to the network, although it is possible to use your own producers

   146    in more complex schemes to implement encryption and chunking, for example.

   147 

   148 

   149 .. method:: async_chat.push_with_producer(producer)

   150 

   151    Takes a producer object and adds it to the producer fifo associated with

   152    the channel.  When all currently-pushed producers have been exhausted the

   153    channel will consume this producer's data by calling its :meth:`more`

   154    method and send the data to the remote endpoint.

   155 

   156 

   157 .. method:: async_chat.readable()

   158 

   159    Should return ``True`` for the channel to be included in the set of

   160    channels tested by the :cfunc:`select` loop for readability.

   161 

   162 

   163 .. method:: async_chat.refill_buffer()

   164 

   165    Refills the output buffer by calling the :meth:`more` method of the

   166    producer at the head of the fifo.  If it is exhausted then the producer is

   167    popped off the fifo and the next producer is activated.  If the current

   168    producer is, or becomes, ``None`` then the channel is closed.

   169 

   170 

   171 .. method:: async_chat.set_terminator(term)

   172 

   173    Sets the terminating condition to be recognized on the channel.  ``term``

   174    may be any of three types of value, corresponding to three different ways

   175    to handle incoming protocol data.

   176 

   177    +-----------+---------------------------------------------+

   178    | term      | Description                                 |

   179    +===========+=============================================+

   180    | *string*  | Will call :meth:`found_terminator` when the |

   181    |           | string is found in the input stream         |

   182    +-----------+---------------------------------------------+

   183    | *integer* | Will call :meth:`found_terminator` when the |

   184    |           | indicated number of characters have been    |

   185    |           | received                                    |

   186    +-----------+---------------------------------------------+

   187    | ``None``  | The channel continues to collect data       |

   188    |           | forever                                     |

   189    +-----------+---------------------------------------------+

   190 

   191    Note that any data following the terminator will be available for reading

   192    by the channel after :meth:`found_terminator` is called.

   193 

   194 

   195 .. method:: async_chat.writable()

   196 

   197    Should return ``True`` as long as items remain on the producer fifo, or the

   198    channel is connected and the channel's output buffer is non-empty.

   199 

   200 

   201 asynchat - Auxiliary Classes and Functions

   202 ------------------------------------------

   203 

   204 

   205 .. class:: simple_producer(data[, buffer_size=512])

   206 

   207    A :class:`simple_producer` takes a chunk of data and an optional buffer

   208    size.  Repeated calls to its :meth:`more` method yield successive chunks of

   209    the data no larger than *buffer_size*.

   210 

   211 

   212    .. method:: more()

   213 

   214       Produces the next chunk of information from the producer, or returns the

   215       empty string.

   216 

   217 

   218 .. class:: fifo([list=None])

   219 

   220    Each channel maintains a :class:`fifo` holding data which has been pushed

   221    by the application but not yet popped for writing to the channel.  A

   222    :class:`fifo` is a list used to hold data and/or producers until they are

   223    required.  If the *list* argument is provided then it should contain

   224    producers or data items to be written to the channel.

   225 

   226 

   227    .. method:: is_empty()

   228 

   229       Returns ``True`` if and only if the fifo is empty.

   230 

   231 

   232    .. method:: first()

   233 

   234       Returns the least-recently :meth:`push`\ ed item from the fifo.

   235 

   236 

   237    .. method:: push(data)

   238 

   239       Adds the given data (which may be a string or a producer object) to the

   240       producer fifo.

   241 

   242 

   243    .. method:: pop()

   244 

   245       If the fifo is not empty, returns ``True, first()``, deleting the popped

   246       item.  Returns ``False, None`` for an empty fifo.

   247 

   248 The :mod:`asynchat` module also defines one utility function, which may be of

   249 use in network and textual analysis operations.

   250 

   251 

   252 .. function:: find_prefix_at_end(haystack, needle)

   253 

   254    Returns ``True`` if string *haystack* ends with any non-empty prefix of

   255    string *needle*.

   256 

   257 

   258 .. _asynchat-example:

   259 

   260 asynchat Example

   261 ----------------

   262 

   263 The following partial example shows how HTTP requests can be read with

   264 :class:`async_chat`.  A web server might create an

   265 :class:`http_request_handler` object for each incoming client connection.

   266 Notice that initially the channel terminator is set to match the blank line at

   267 the end of the HTTP headers, and a flag indicates that the headers are being

   268 read.

   269 

   270 Once the headers have been read, if the request is of type POST (indicating

   271 that further data are present in the input stream) then the

   272 ``Content-Length:`` header is used to set a numeric terminator to read the

   273 right amount of data from the channel.

   274 

   275 The :meth:`handle_request` method is called once all relevant input has been

   276 marshalled, after setting the channel terminator to ``None`` to ensure that

   277 any extraneous data sent by the web client are ignored. ::

   278 

   279    class http_request_handler(asynchat.async_chat):

   280 

   281        def __init__(self, sock, addr, sessions, log):

   282            asynchat.async_chat.__init__(self, sock=sock)

   283            self.addr = addr

   284            self.sessions = sessions

   285            self.ibuffer = []

   286            self.obuffer = ""

   287            self.set_terminator("\r\n\r\n")

   288            self.reading_headers = True

   289            self.handling = False

   290            self.cgi_data = None

   291            self.log = log

   292 

   293        def collect_incoming_data(self, data):

   294            """Buffer the data"""

   295            self.ibuffer.append(data)

   296 

   297        def found_terminator(self):

   298            if self.reading_headers:

   299                self.reading_headers = False

   300                self.parse_headers("".join(self.ibuffer))

   301                self.ibuffer = []

   302                if self.op.upper() == "POST":

   303                    clen = self.headers.getheader("content-length")

   304                    self.set_terminator(int(clen))

   305                else:

   306                    self.handling = True

   307                    self.set_terminator(None)

   308                    self.handle_request()

   309            elif not self.handling:

   310                self.set_terminator(None) # browsers sometimes over-send

   311                self.cgi_data = parse(self.headers, "".join(self.ibuffer))

   312                self.handling = True

   313                self.ibuffer = []

   314                self.handle_request()