.. highlightlang:: none

.. _install-index:

*****************************

  Installing Python Modules  

*****************************

:Author: Greg Ward

:Release: |version|

:Date: |today|

.. TODO: Fill in XXX comments

.. The audience for this document includes people who don't know anything

   about Python and aren't about to learn the language just in order to

   install and maintain it for their users, i.e. system administrators.

   Thus, I have to be sure to explain the basics at some point:

   sys.path and PYTHONPATH at least.  Should probably give pointers to

   other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.

   Finally, it might be useful to include all the material from my "Care

   and Feeding of a Python Installation" talk in here somewhere.  Yow!

.. topic:: Abstract

   This document describes the Python Distribution Utilities ("Distutils") from the

   end-user's point-of-view, describing how to extend the capabilities of a

   standard Python installation by building and installing third-party Python

   modules and extensions.

.. _inst-intro:

Introduction

============

Although Python's extensive standard library covers many programming needs,

there often comes a time when you need to add some new functionality to your

Python installation in the form of third-party modules.  This might be necessary

to support your own programming, or to support an application that you want to

use and that happens to be written in Python.

In the past, there has been little support for adding third-party modules to an

existing Python installation.  With the introduction of the Python Distribution

Utilities (Distutils for short) in Python 2.0, this changed.

This document is aimed primarily at the people who need to install third-party

Python modules: end-users and system administrators who just need to get some

Python application running, and existing Python programmers who want to add some

new goodies to their toolbox.  You don't need to know Python to read this

document; there will be some brief forays into using Python's interactive mode

to explore your installation, but that's it.  If you're looking for information

on how to distribute your own Python modules so that others may use them, see

the :ref:`distutils-index` manual.

.. _inst-trivial-install:

Best case: trivial installation

-------------------------------

In the best case, someone will have prepared a special version of the module

distribution you want to install that is targeted specifically at your platform

and is installed just like any other software on your platform.  For example,

the module developer might make an executable installer available for Windows

users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE,

Mandrake, and many others), a Debian package for users of Debian-based Linux

systems, and so forth.

In that case, you would download the installer appropriate to your platform and

do the obvious thing with it: run it if it's an executable installer, ``rpm

--install`` it if it's an RPM, etc.  You don't need to run Python or a setup

script, you don't need to compile anything---you might not even need to read any

instructions (although it's always a good idea to do so anyways).

Of course, things will not always be that easy.  You might be interested in a

module distribution that doesn't have an easy-to-use installer for your

platform.  In that case, you'll have to start with the source distribution

released by the module's author/maintainer.  Installing from a source

distribution is not too hard, as long as the modules are packaged in the

standard way.  The bulk of this document is about building and installing

modules from standard source distributions.

.. _inst-new-standard:

The new standard: Distutils

---------------------------

If you download a module source distribution, you can tell pretty quickly if it

was packaged and distributed in the standard way, i.e. using the Distutils.

First, the distribution's name and version number will be featured prominently

in the name of the downloaded archive, e.g. :file:`foo-1.0.tar.gz` or

:file:`widget-0.9.7.zip`.  Next, the archive will unpack into a similarly-named

directory: :file:`foo-1.0` or :file:`widget-0.9.7`.  Additionally, the

distribution will contain a setup script :file:`setup.py`, and a file named

:file:`README.txt` or possibly just :file:`README`, which should explain that

building and installing the module distribution is a simple matter of running ::

   python setup.py install

If all these things are true, then you already know how to build and install the

modules you've just downloaded:  Run the command above. Unless you need to

install things in a non-standard way or customize the build process, you don't

really need this manual.  Or rather, the above command is everything you need to

get out of this manual.

.. _inst-standard-install:

Standard Build and Install

==========================

As described in section :ref:`inst-new-standard`, building and installing a module

distribution using the Distutils is usually one simple command::

   python setup.py install

On Unix, you'd run this command from a shell prompt; on Windows, you have to

open a command prompt window ("DOS box") and do it there; on Mac OS X, you open

a :command:`Terminal` window to get a shell prompt.

.. _inst-platform-variations:

Platform variations

-------------------

You should always run the setup command from the distribution root directory,

i.e. the top-level subdirectory that the module source distribution unpacks

into.  For example, if you've just downloaded a module source distribution

:file:`foo-1.0.tar.gz` onto a Unix system, the normal thing to do is::

   gunzip -c foo-1.0.tar.gz | tar xf -    # unpacks into directory foo-1.0

   cd foo-1.0

   python setup.py install

On Windows, you'd probably download :file:`foo-1.0.zip`.  If you downloaded the

archive file to :file:`C:\\Temp`, then it would unpack into

:file:`C:\\Temp\\foo-1.0`; you can use either a archive manipulator with a

graphical user interface (such as WinZip) or a command-line tool (such as

:program:`unzip` or :program:`pkunzip`) to unpack the archive.  Then, open a

command prompt window ("DOS box"), and run::

   cd c:\Temp\foo-1.0

   python setup.py install

.. _inst-splitting-up:

Splitting the job up

--------------------

Running ``setup.py install`` builds and installs all modules in one run.  If you

prefer to work incrementally---especially useful if you want to customize the

build process, or if things are going wrong---you can use the setup script to do

one thing at a time.  This is particularly helpful when the build and install

will be done by different users---for example, you might want to build a module

distribution and hand it off to a system administrator for installation (or do

it yourself, with super-user privileges).

For example, you can build everything in one step, and then install everything

in a second step, by invoking the setup script twice::

   python setup.py build

   python setup.py install

If you do this, you will notice that running the :command:`install` command

first runs the :command:`build` command, which---in this case---quickly notices

that it has nothing to do, since everything in the :file:`build` directory is

up-to-date.

You may not need this ability to break things down often if all you do is

install modules downloaded off the 'net, but it's very handy for more advanced

tasks.  If you get into distributing your own Python modules and extensions,

you'll run lots of individual Distutils commands on their own.

.. _inst-how-build-works:

How building works

------------------

As implied above, the :command:`build` command is responsible for putting the

files to install into a *build directory*.  By default, this is :file:`build`

under the distribution root; if you're excessively concerned with speed, or want

to keep the source tree pristine, you can change the build directory with the

:option:`--build-base` option. For example::

   python setup.py build --build-base=/tmp/pybuild/foo-1.0

(Or you could do this permanently with a directive in your system or personal

Distutils configuration file; see section :ref:`inst-config-files`.)  Normally, this

isn't necessary.

The default layout for the build tree is as follows::

   --- build/ --- lib/

   or

   --- build/ --- lib.<plat>/

                  temp.<plat>/

where ``<plat>`` expands to a brief description of the current OS/hardware

platform and Python version.  The first form, with just a :file:`lib` directory,

is used for "pure module distributions"---that is, module distributions that

include only pure Python modules.  If a module distribution contains any

extensions (modules written in C/C++), then the second form, with two ``<plat>``

directories, is used.  In that case, the :file:`temp.{plat}` directory holds

temporary files generated by the compile/link process that don't actually get

installed.  In either case, the :file:`lib` (or :file:`lib.{plat}`) directory

contains all Python modules (pure Python and extensions) that will be installed.

In the future, more directories will be added to handle Python scripts,

documentation, binary executables, and whatever else is needed to handle the job

of installing Python modules and applications.

.. _inst-how-install-works:

How installation works

----------------------

After the :command:`build` command runs (whether you run it explicitly, or the

:command:`install` command does it for you), the work of the :command:`install`

command is relatively simple: all it has to do is copy everything under

:file:`build/lib` (or :file:`build/lib.{plat}`) to your chosen installation

directory.

If you don't choose an installation directory---i.e., if you just run ``setup.py

install``\ ---then the :command:`install` command installs to the standard

location for third-party Python modules.  This location varies by platform and

by how you built/installed Python itself.  On Unix (and Mac OS X, which is also

Unix-based), it also depends on whether the module distribution being installed

is pure Python or contains extensions ("non-pure"):

+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+

| Platform        | Standard installation location                      | Default value                                    | Notes |

+=================+=====================================================+==================================================+=======+

| Unix (pure)     | :file:`{prefix}/lib/python{X.Y}/site-packages`      | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1)  |

+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+

| Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1)  |

+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+

| Windows         | :file:`{prefix}`                                    | :file:`C:\\Python`                               | \(2)  |

+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+

Notes:

(1)

   Most Linux distributions include Python as a standard part of the system, so

   :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on

   Linux.  If you build Python yourself on Linux (or any Unix-like system), the

   default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`.

(2)

   The default installation directory on Windows was :file:`C:\\Program

   Files\\Python` under Python 1.6a1, 1.5.2, and earlier.

:file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python

is installed to, and where it finds its libraries at run-time.  They are always

the same under Windows, and very often the same under Unix and Mac OS X.  You

can find out what your Python installation uses for :file:`{prefix}` and

:file:`{exec-prefix}` by running Python in interactive mode and typing a few

simple commands. Under Unix, just type ``python`` at the shell prompt.  Under

Windows, choose :menuselection:`Start --> Programs --> Python X.Y -->

Python (command line)`.   Once the interpreter is started, you type Python code

at the prompt.  For example, on my Linux system, I type the three Python

statements shown below, and get the output as shown, to find out my

:file:`{prefix}` and :file:`{exec-prefix}`::

   Python 2.4 (#26, Aug  7 2004, 17:19:02) 

   Type "help", "copyright", "credits" or "license" for more information.

   >>> import sys

   >>> sys.prefix

   '/usr'

   >>> sys.exec_prefix

   '/usr'

If you don't want to install modules to the standard location, or if you don't

have permission to write there, then you need to read about alternate

installations in section :ref:`inst-alt-install`.  If you want to customize your

installation directories more heavily, see section :ref:`inst-custom-install` on

custom installations.

.. _inst-alt-install:

Alternate Installation

======================

Often, it is necessary or desirable to install modules to a location other than

the standard location for third-party Python modules.  For example, on a Unix

system you might not have permission to write to the standard third-party module

directory.  Or you might wish to try out a module before making it a standard

part of your local Python installation.  This is especially true when upgrading

a distribution already present: you want to make sure your existing base of

scripts still works with the new version before actually upgrading.

The Distutils :command:`install` command is designed to make installing module

distributions to an alternate location simple and painless.  The basic idea is

that you supply a base directory for the installation, and the

:command:`install` command picks a set of directories (called an *installation

scheme*) under this base directory in which to install files.  The details

differ across platforms, so read whichever of the following sections applies to

you.

.. _inst-alt-install-prefix:

Alternate installation: the home scheme

---------------------------------------

The idea behind the "home scheme" is that you build and maintain a personal

stash of Python modules.  This scheme's name is derived from the idea of a

"home" directory on Unix, since it's not unusual for a Unix user to make their

home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`.

This scheme can be used by anyone, regardless of the operating system their

installing for.

Installing a new module distribution is as simple as ::

   python setup.py install --home=<dir>

where you can supply any directory you like for the :option:`--home` option.  On

Unix, lazy typists can just type a tilde (``~``); the :command:`install` command

will expand this to your home directory::

   python setup.py install --home=~

The :option:`--home` option defines the installation base directory.  Files are

installed to the following directories under the installation base as follows:

+------------------------------+---------------------------+-----------------------------+

| Type of file                 | Installation Directory    | Override option             |

+==============================+===========================+=============================+

| pure module distribution     | :file:`{home}/lib/python` | :option:`--install-purelib` |

+------------------------------+---------------------------+-----------------------------+

| non-pure module distribution | :file:`{home}/lib/python` | :option:`--install-platlib` |

+------------------------------+---------------------------+-----------------------------+

| scripts                      | :file:`{home}/bin`        | :option:`--install-scripts` |

+------------------------------+---------------------------+-----------------------------+

| data                         | :file:`{home}/share`      | :option:`--install-data`    |

+------------------------------+---------------------------+-----------------------------+

.. versionchanged:: 2.4

   The :option:`--home` option used to be supported only on Unix.

.. _inst-alt-install-home:

Alternate installation: Unix (the prefix scheme)

------------------------------------------------

The "prefix scheme" is useful when you wish to use one Python installation to

perform the build/install (i.e., to run the setup script), but install modules

into the third-party module directory of a different Python installation (or

something that looks like a different Python installation).  If this sounds a

trifle unusual, it is---that's why the "home scheme" comes first.  However,

there are at least two known cases where the prefix scheme will be useful.

First, consider that many Linux distributions put Python in :file:`/usr`, rather

than the more traditional :file:`/usr/local`.  This is entirely appropriate,

since in those cases Python is part of "the system" rather than a local add-on.

However, if you are installing Python modules from source, you probably want

them to go in :file:`/usr/local/lib/python2.{X}` rather than

:file:`/usr/lib/python2.{X}`.  This can be done with ::

   /usr/bin/python setup.py install --prefix=/usr/local

Another possibility is a network filesystem where the name used to write to a

remote directory is different from the name used to read it: for example, the

Python interpreter accessed as :file:`/usr/local/bin/python` might search for

modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to

be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`.  This could

be done with ::

   /usr/local/bin/python setup.py install --prefix=/mnt/@server/export

In either case, the :option:`--prefix` option defines the installation base, and

the :option:`--exec-prefix` option defines the platform-specific installation

base, which is used for platform-specific files.  (Currently, this just means

non-pure module distributions, but could be expanded to C libraries, binary

executables, etc.)  If :option:`--exec-prefix` is not supplied, it defaults to

:option:`--prefix`.  Files are installed as follows:

+------------------------------+-----------------------------------------------------+-----------------------------+

| Type of file                 | Installation Directory                              | Override option             |

+==============================+=====================================================+=============================+

| pure module distribution     | :file:`{prefix}/lib/python{X.Y}/site-packages`      | :option:`--install-purelib` |

+------------------------------+-----------------------------------------------------+-----------------------------+

| non-pure module distribution | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :option:`--install-platlib` |

+------------------------------+-----------------------------------------------------+-----------------------------+

| scripts                      | :file:`{prefix}/bin`                                | :option:`--install-scripts` |

+------------------------------+-----------------------------------------------------+-----------------------------+

| data                         | :file:`{prefix}/share`                              | :option:`--install-data`    |

+------------------------------+-----------------------------------------------------+-----------------------------+

There is no requirement that :option:`--prefix` or :option:`--exec-prefix`

actually point to an alternate Python installation; if the directories listed

above do not already exist, they are created at installation time.

Incidentally, the real reason the prefix scheme is important is simply that a

standard Unix installation uses the prefix scheme, but with :option:`--prefix`

and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and

``sys.exec_prefix``.  Thus, you might think you'll never use the prefix scheme,

but every time you run ``python setup.py install`` without any other options,

you're using it.

Note that installing extensions to an alternate Python installation has no

effect on how those extensions are built: in particular, the Python header files

(:file:`Python.h` and friends) installed with the Python interpreter used to run

the setup script will be used in compiling extensions.  It is your

responsibility to ensure that the interpreter used to run extensions installed

in this way is compatible with the interpreter used to build them.  The best way

to do this is to ensure that the two interpreters are the same version of Python

(possibly different builds, or possibly copies of the same build).  (Of course,

if your :option:`--prefix` and :option:`--exec-prefix` don't even point to an

alternate Python installation, this is immaterial.)

.. _inst-alt-install-windows:

Alternate installation: Windows (the prefix scheme)

---------------------------------------------------

Windows has no concept of a user's home directory, and since the standard Python

installation under Windows is simpler than under Unix, the :option:`--prefix`

option has traditionally been used to install additional packages in separate

locations on Windows. ::

   python setup.py install --prefix="\Temp\Python"

to install modules to the :file:`\\Temp\\Python` directory on the current drive.

The installation base is defined by the :option:`--prefix` option; the

:option:`--exec-prefix` option is not supported under Windows. Files are

installed as follows:

+------------------------------+---------------------------+-----------------------------+

| Type of file                 | Installation Directory    | Override option             |

+==============================+===========================+=============================+

| pure module distribution     | :file:`{prefix}`          | :option:`--install-purelib` |

+------------------------------+---------------------------+-----------------------------+

| non-pure module distribution | :file:`{prefix}`          | :option:`--install-platlib` |

+------------------------------+---------------------------+-----------------------------+

| scripts                      | :file:`{prefix}\\Scripts` | :option:`--install-scripts` |

+------------------------------+---------------------------+-----------------------------+

| data                         | :file:`{prefix}\\Data`    | :option:`--install-data`    |

+------------------------------+---------------------------+-----------------------------+

.. _inst-custom-install:

Custom Installation

===================

Sometimes, the alternate installation schemes described in section

:ref:`inst-alt-install` just don't do what you want.  You might want to tweak just

one or two directories while keeping everything under the same base directory,

or you might want to completely redefine the installation scheme.  In either

case, you're creating a *custom installation scheme*.

You probably noticed the column of "override options" in the tables describing

the alternate installation schemes above.  Those options are how you define a

custom installation scheme.  These override options can be relative, absolute,

or explicitly defined in terms of one of the installation base directories.

(There are two installation base directories, and they are normally the same---

they only differ when you use the Unix "prefix scheme" and supply different

:option:`--prefix` and :option:`--exec-prefix` options.)

For example, say you're installing a module distribution to your home directory

under Unix---but you want scripts to go in :file:`~/scripts` rather than

:file:`~/bin`. As you might expect, you can override this directory with the

:option:`--install-scripts` option; in this case, it makes most sense to supply

a relative path, which will be interpreted relative to the installation base

directory (your home directory, in this case)::

   python setup.py install --home=~ --install-scripts=scripts

Another Unix example: suppose your Python installation was built and installed

with a prefix of :file:`/usr/local/python`, so under a standard  installation

scripts will wind up in :file:`/usr/local/python/bin`.  If you want them in

:file:`/usr/local/bin` instead, you would supply this absolute directory for the

:option:`--install-scripts` option::

   python setup.py install --install-scripts=/usr/local/bin

(This performs an installation using the "prefix scheme," where the prefix is

whatever your Python interpreter was installed with--- :file:`/usr/local/python`

in this case.)

If you maintain Python on Windows, you might want third-party modules to live in

a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}`

itself.  This is almost as easy as customizing the script installation directory

---you just have to remember that there are two types of modules to worry about,

pure modules and non-pure modules (i.e., modules from a non-pure distribution).

For example::

   python setup.py install --install-purelib=Site --install-platlib=Site

The specified installation directories are relative to :file:`{prefix}`.  Of

course, you also have to ensure that these directories are in Python's module

search path, such as by putting a :file:`.pth` file in :file:`{prefix}`.  See

section :ref:`inst-search-path` to find out how to modify Python's search path.

If you want to define an entire installation scheme, you just have to supply all

of the installation directory options.  The recommended way to do this is to

supply relative paths; for example, if you want to maintain all Python

module-related files under :file:`python` in your home directory, and you want a

separate directory for each platform that you use your home directory from, you

might define the following installation scheme::

   python setup.py install --home=~ \

                           --install-purelib=python/lib \

                           --install-platlib=python/lib.$PLAT \

                           --install-scripts=python/scripts

                           --install-data=python/data

or, equivalently, ::

   python setup.py install --home=~/python \

                           --install-purelib=lib \

                           --install-platlib='lib.$PLAT' \

                           --install-scripts=scripts

                           --install-data=data

``$PLAT`` is not (necessarily) an environment variable---it will be expanded by

the Distutils as it parses your command line options, just as it does when