:mod:`xml.parsers.expat` --- Fast XML parsing using Expat

=========================================================

.. module:: xml.parsers.expat

   :synopsis: An interface to the Expat non-validating XML parser.

.. moduleauthor:: Paul Prescod <paul@prescod.net>

.. Markup notes:

   Many of the attributes of the XMLParser objects are callbacks.  Since

   signature information must be presented, these are described using the method

   directive.  Since they are attributes which are set by client code, in-text

   references to these attributes should be marked using the :member: role.

.. versionadded:: 2.0

.. index:: single: Expat

The :mod:`xml.parsers.expat` module is a Python interface to the Expat

non-validating XML parser. The module provides a single extension type,

:class:`xmlparser`, that represents the current state of an XML parser.  After

an :class:`xmlparser` object has been created, various attributes of the object

can be set to handler functions.  When an XML document is then fed to the

parser, the handler functions are called for the character data and markup in

the XML document.

.. index:: module: pyexpat

This module uses the :mod:`pyexpat` module to provide access to the Expat

parser.  Direct use of the :mod:`pyexpat` module is deprecated.

This module provides one exception and one type object:

.. exception:: ExpatError

   The exception raised when Expat reports an error.  See section

   :ref:`expaterror-objects` for more information on interpreting Expat errors.

.. exception:: error

   Alias for :exc:`ExpatError`.

.. data:: XMLParserType

   The type of the return values from the :func:`ParserCreate` function.

The :mod:`xml.parsers.expat` module contains two functions:

.. function:: ErrorString(errno)

   Returns an explanatory string for a given error number *errno*.

.. function:: ParserCreate([encoding[, namespace_separator]])

   Creates and returns a new :class:`xmlparser` object.   *encoding*, if specified,

   must be a string naming the encoding  used by the XML data.  Expat doesn't

   support as many encodings as Python does, and its repertoire of encodings can't

   be extended; it supports UTF-8, UTF-16, ISO-8859-1 (Latin1), and ASCII.  If

   *encoding* [1]_ is given it will override the implicit or explicit encoding of the

   document.

   Expat can optionally do XML namespace processing for you, enabled by providing a

   value for *namespace_separator*.  The value must be a one-character string; a

   :exc:`ValueError` will be raised if the string has an illegal length (``None``

   is considered the same as omission).  When namespace processing is enabled,

   element type names and attribute names that belong to a namespace will be

   expanded.  The element name passed to the element handlers

   :attr:`StartElementHandler` and :attr:`EndElementHandler` will be the

   concatenation of the namespace URI, the namespace separator character, and the

   local part of the name.  If the namespace separator is a zero byte (``chr(0)``)

   then the namespace URI and the local part will be concatenated without any

   separator.

   For example, if *namespace_separator* is set to a space character (``' '``) and

   the following document is parsed::

      <?xml version="1.0"?>

      <root xmlns    = "http://default-namespace.org/"

            xmlns:py = "http://www.python.org/ns/">

        <py:elem1 />

        <elem2 xmlns="" />

      </root>

   :attr:`StartElementHandler` will receive the following strings for each

   element::

      http://default-namespace.org/ root

      http://www.python.org/ns/ elem1

      elem2

.. seealso::

   `The Expat XML Parser <http://www.libexpat.org/>`_

      Home page of the Expat project.

.. _xmlparser-objects:

XMLParser Objects

-----------------

:class:`xmlparser` objects have the following methods:

.. method:: xmlparser.Parse(data[, isfinal])

   Parses the contents of the string *data*, calling the appropriate handler

   functions to process the parsed data.  *isfinal* must be true on the final call

   to this method.  *data* can be the empty string at any time.

.. method:: xmlparser.ParseFile(file)

   Parse XML data reading from the object *file*.  *file* only needs to provide

   the ``read(nbytes)`` method, returning the empty string when there's no more

   data.

.. method:: xmlparser.SetBase(base)

   Sets the base to be used for resolving relative URIs in system identifiers in

   declarations.  Resolving relative identifiers is left to the application: this

   value will be passed through as the *base* argument to the

   :func:`ExternalEntityRefHandler`, :func:`NotationDeclHandler`, and

   :func:`UnparsedEntityDeclHandler` functions.

.. method:: xmlparser.GetBase()

   Returns a string containing the base set by a previous call to :meth:`SetBase`,

   or ``None`` if  :meth:`SetBase` hasn't been called.

.. method:: xmlparser.GetInputContext()

   Returns the input data that generated the current event as a string. The data is

   in the encoding of the entity which contains the text. When called while an

   event handler is not active, the return value is ``None``.

   .. versionadded:: 2.1

.. method:: xmlparser.ExternalEntityParserCreate(context[, encoding])

   Create a "child" parser which can be used to parse an external parsed entity

   referred to by content parsed by the parent parser.  The *context* parameter

   should be the string passed to the :meth:`ExternalEntityRefHandler` handler

   function, described below. The child parser is created with the

   :attr:`ordered_attributes`, :attr:`returns_unicode` and

   :attr:`specified_attributes` set to the values of this parser.

.. method:: xmlparser.UseForeignDTD([flag])

   Calling this with a true value for *flag* (the default) will cause Expat to call

   the :attr:`ExternalEntityRefHandler` with :const:`None` for all arguments to

   allow an alternate DTD to be loaded.  If the document does not contain a

   document type declaration, the :attr:`ExternalEntityRefHandler` will still be

   called, but the :attr:`StartDoctypeDeclHandler` and

   :attr:`EndDoctypeDeclHandler` will not be called.

   Passing a false value for *flag* will cancel a previous call that passed a true

   value, but otherwise has no effect.

   This method can only be called before the :meth:`Parse` or :meth:`ParseFile`

   methods are called; calling it after either of those have been called causes

   :exc:`ExpatError` to be raised with the :attr:`code` attribute set to

   :const:`errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING`.

   .. versionadded:: 2.3

:class:`xmlparser` objects have the following attributes:

.. attribute:: xmlparser.buffer_size

   The size of the buffer used when :attr:`buffer_text` is true.  

   A new buffer size can be set by assigning a new integer value 

   to this attribute.  

   When the size is changed, the buffer will be flushed.

   .. versionadded:: 2.3

   .. versionchanged:: 2.6

      The buffer size can now be changed.

.. attribute:: xmlparser.buffer_text

   Setting this to true causes the :class:`xmlparser` object to buffer textual

   content returned by Expat to avoid multiple calls to the

   :meth:`CharacterDataHandler` callback whenever possible.  This can improve

   performance substantially since Expat normally breaks character data into chunks

   at every line ending.  This attribute is false by default, and may be changed at

   any time.

   .. versionadded:: 2.3

.. attribute:: xmlparser.buffer_used

   If :attr:`buffer_text` is enabled, the number of bytes stored in the buffer.

   These bytes represent UTF-8 encoded text.  This attribute has no meaningful

   interpretation when :attr:`buffer_text` is false.

   .. versionadded:: 2.3

.. attribute:: xmlparser.ordered_attributes

   Setting this attribute to a non-zero integer causes the attributes to be

   reported as a list rather than a dictionary.  The attributes are presented in

   the order found in the document text.  For each attribute, two list entries are

   presented: the attribute name and the attribute value.  (Older versions of this

   module also used this format.)  By default, this attribute is false; it may be

   changed at any time.

   .. versionadded:: 2.1

.. attribute:: xmlparser.returns_unicode

   If this attribute is set to a non-zero integer, the handler functions will be

   passed Unicode strings.  If :attr:`returns_unicode` is :const:`False`, 8-bit

   strings containing UTF-8 encoded data will be passed to the handlers.  This is

   :const:`True` by default when Python is built with Unicode support.

   .. versionchanged:: 1.6

      Can be changed at any time to affect the result type.

.. attribute:: xmlparser.specified_attributes

   If set to a non-zero integer, the parser will report only those attributes which

   were specified in the document instance and not those which were derived from

   attribute declarations.  Applications which set this need to be especially

   careful to use what additional information is available from the declarations as

   needed to comply with the standards for the behavior of XML processors.  By

   default, this attribute is false; it may be changed at any time.

   .. versionadded:: 2.1

The following attributes contain values relating to the most recent error

encountered by an :class:`xmlparser` object, and will only have correct values

once a call to :meth:`Parse` or :meth:`ParseFile` has raised a

:exc:`xml.parsers.expat.ExpatError` exception.

.. attribute:: xmlparser.ErrorByteIndex

   Byte index at which an error occurred.

.. attribute:: xmlparser.ErrorCode

   Numeric code specifying the problem.  This value can be passed to the

   :func:`ErrorString` function, or compared to one of the constants defined in the

   ``errors`` object.

.. attribute:: xmlparser.ErrorColumnNumber

   Column number at which an error occurred.

.. attribute:: xmlparser.ErrorLineNumber

   Line number at which an error occurred.

The following attributes contain values relating to the current parse location

in an :class:`xmlparser` object.  During a callback reporting a parse event they

indicate the location of the first of the sequence of characters that generated

the event.  When called outside of a callback, the position indicated will be

just past the last parse event (regardless of whether there was an associated

callback).

.. versionadded:: 2.4

.. attribute:: xmlparser.CurrentByteIndex

   Current byte index in the parser input.

.. attribute:: xmlparser.CurrentColumnNumber

   Current column number in the parser input.

.. attribute:: xmlparser.CurrentLineNumber

   Current line number in the parser input.

Here is the list of handlers that can be set.  To set a handler on an

:class:`xmlparser` object *o*, use ``o.handlername = func``.  *handlername* must

be taken from the following list, and *func* must be a callable object accepting

the correct number of arguments.  The arguments are all strings, unless

otherwise stated.

.. method:: xmlparser.XmlDeclHandler(version, encoding, standalone)

   Called when the XML declaration is parsed.  The XML declaration is the

   (optional) declaration of the applicable version of the XML recommendation, the

   encoding of the document text, and an optional "standalone" declaration.

   *version* and *encoding* will be strings of the type dictated by the

   :attr:`returns_unicode` attribute, and *standalone* will be ``1`` if the

   document is declared standalone, ``0`` if it is declared not to be standalone,

   or ``-1`` if the standalone clause was omitted. This is only available with

   Expat version 1.95.0 or newer.

   .. versionadded:: 2.1

.. method:: xmlparser.StartDoctypeDeclHandler(doctypeName, systemId, publicId, has_internal_subset)

   Called when Expat begins parsing the document type declaration (``<!DOCTYPE

   ...``).  The *doctypeName* is provided exactly as presented.  The *systemId* and

   *publicId* parameters give the system and public identifiers if specified, or

   ``None`` if omitted.  *has_internal_subset* will be true if the document

   contains and internal document declaration subset. This requires Expat version

   1.2 or newer.

.. method:: xmlparser.EndDoctypeDeclHandler()

   Called when Expat is done parsing the document type declaration. This requires

   Expat version 1.2 or newer.

.. method:: xmlparser.ElementDeclHandler(name, model)

   Called once for each element type declaration.  *name* is the name of the

   element type, and *model* is a representation of the content model.

.. method:: xmlparser.AttlistDeclHandler(elname, attname, type, default, required)

   Called for each declared attribute for an element type.  If an attribute list

   declaration declares three attributes, this handler is called three times, once

   for each attribute.  *elname* is the name of the element to which the

   declaration applies and *attname* is the name of the attribute declared.  The

   attribute type is a string passed as *type*; the possible values are

   ``'CDATA'``, ``'ID'``, ``'IDREF'``, ... *default* gives the default value for

   the attribute used when the attribute is not specified by the document instance,

   or ``None`` if there is no default value (``#IMPLIED`` values).  If the

   attribute is required to be given in the document instance, *required* will be

   true. This requires Expat version 1.95.0 or newer.

.. method:: xmlparser.StartElementHandler(name, attributes)

   Called for the start of every element.  *name* is a string containing the

   element name, and *attributes* is a dictionary mapping attribute names to their

   values.

.. method:: xmlparser.EndElementHandler(name)

   Called for the end of every element.

.. method:: xmlparser.ProcessingInstructionHandler(target, data)

   Called for every processing instruction.

.. method:: xmlparser.CharacterDataHandler(data)

   Called for character data.  This will be called for normal character data, CDATA

   marked content, and ignorable whitespace.  Applications which must distinguish

   these cases can use the :attr:`StartCdataSectionHandler`,

   :attr:`EndCdataSectionHandler`, and :attr:`ElementDeclHandler` callbacks to

   collect the required information.

.. method:: xmlparser.UnparsedEntityDeclHandler(entityName, base, systemId, publicId, notationName)

   Called for unparsed (NDATA) entity declarations.  This is only present for

   version 1.2 of the Expat library; for more recent versions, use

   :attr:`EntityDeclHandler` instead.  (The underlying function in the Expat

   library has been declared obsolete.)

.. method:: xmlparser.EntityDeclHandler(entityName, is_parameter_entity, value, base, systemId, publicId, notationName)

   Called for all entity declarations.  For parameter and internal entities,

   *value* will be a string giving the declared contents of the entity; this will

   be ``None`` for external entities.  The *notationName* parameter will be

   ``None`` for parsed entities, and the name of the notation for unparsed

   entities. *is_parameter_entity* will be true if the entity is a parameter entity

   or false for general entities (most applications only need to be concerned with

   general entities). This is only available starting with version 1.95.0 of the

   Expat library.

   .. versionadded:: 2.1

.. method:: xmlparser.NotationDeclHandler(notationName, base, systemId, publicId)

   Called for notation declarations.  *notationName*, *base*, and *systemId*, and

   *publicId* are strings if given.  If the public identifier is omitted,

   *publicId* will be ``None``.

.. method:: xmlparser.StartNamespaceDeclHandler(prefix, uri)

   Called when an element contains a namespace declaration.  Namespace declarations

   are processed before the :attr:`StartElementHandler` is called for the element

   on which declarations are placed.

.. method:: xmlparser.EndNamespaceDeclHandler(prefix)

   Called when the closing tag is reached for an element  that contained a

   namespace declaration.  This is called once for each namespace declaration on

   the element in the reverse of the order for which the

   :attr:`StartNamespaceDeclHandler` was called to indicate the start of each

   namespace declaration's scope.  Calls to this handler are made after the

   corresponding :attr:`EndElementHandler` for the end of the element.

.. method:: xmlparser.CommentHandler(data)

   Called for comments.  *data* is the text of the comment, excluding the leading

   '``<!-``\ ``-``' and trailing '``-``\ ``->``'.

.. method:: xmlparser.StartCdataSectionHandler()

   Called at the start of a CDATA section.  This and :attr:`EndCdataSectionHandler`

   are needed to be able to identify the syntactical start and end for CDATA

   sections.

.. method:: xmlparser.EndCdataSectionHandler()

   Called at the end of a CDATA section.

.. method:: xmlparser.DefaultHandler(data)

   Called for any characters in the XML document for which no applicable handler

   has been specified.  This means characters that are part of a construct which

   could be reported, but for which no handler has been supplied.

.. method:: xmlparser.DefaultHandlerExpand(data)

   This is the same as the :func:`DefaultHandler`,  but doesn't inhibit expansion

   of internal entities. The entity reference will not be passed to the default

   handler.

.. method:: xmlparser.NotStandaloneHandler()

   Called if the XML document hasn't been declared as being a standalone document.

   This happens when there is an external subset or a reference to a parameter

   entity, but the XML declaration does not set standalone to ``yes`` in an XML

   declaration.  If this handler returns ``0``, then the parser will throw an

   :const:`XML_ERROR_NOT_STANDALONE` error.  If this handler is not set, no

   exception is raised by the parser for this condition.

.. method:: xmlparser.ExternalEntityRefHandler(context, base, systemId, publicId)

   Called for references to external entities.  *base* is the current base, as set

   by a previous call to :meth:`SetBase`.  The public and system identifiers,

   *systemId* and *publicId*, are strings if given; if the public identifier is not

   given, *publicId* will be ``None``.  The *context* value is opaque and should

   only be used as described below.

   For external entities to be parsed, this handler must be implemented. It is

   responsible for creating the sub-parser using

   ``ExternalEntityParserCreate(context)``, initializing it with the appropriate

   callbacks, and parsing the entity.  This handler should return an integer; if it

   returns ``0``, the parser will throw an

   :const:`XML_ERROR_EXTERNAL_ENTITY_HANDLING` error, otherwise parsing will

   continue.

   If this handler is not provided, external entities are reported by the

   :attr:`DefaultHandler` callback, if provided.

.. _expaterror-objects:

ExpatError Exceptions

---------------------

.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>

:exc:`ExpatError` exceptions have a number of interesting attributes:

.. attribute:: ExpatError.code

   Expat's internal error number for the specific error.  This will match one of

   the constants defined in the ``errors`` object from this module.

   .. versionadded:: 2.1

.. attribute:: ExpatError.lineno

   Line number on which the error was detected.  The first line is numbered ``1``.

   .. versionadded:: 2.1

.. attribute:: ExpatError.offset

   Character offset into the line where the error occurred.  The first column is

   numbered ``0``.

   .. versionadded:: 2.1

.. _expat-example:

Example