This is a port of Python 2.6 to OS/2 using the EMX development tools

=========================================================================

What's new since the previous release

-------------------------------------

Another day, another version...

Licenses and info about Python and EMX

--------------------------------------

Please read the file README.Python-2.6 included in this package for 

information about Python 2.6.  This file is the README file from the 

Python 2.6 source distribution available via http://www.python.org/ 

and its mirrors.  The file LICENCE.Python-2.6 is the text of the Licence 

from the Python 2.6 source distribution.

Note that the EMX package that this package depends on is released under 

the GNU General Public Licence.  Please refer to the documentation 

accompanying the EMX Runtime libraries for more information about the 

implications of this.  A copy of version 2 of the GPL is included as the 

file COPYING.gpl2.

Readline and GDBM are covered by the GNU General Public Licence.  I think 

Eberhard Mattes' porting changes to BSD DB v1.85 are also GPL'ed (BSD DB 

itself is BSD Licenced).  ncurses and expat appear to be covered by MIT 

style licences - please refer to the source distributions for more detail.  

zlib is distributable under a very free license.  GNU UFC is under the 

GNU LGPL (see file COPYING.lib).

My patches to the Python-2.x source distributions, and any other packages 

used in this port, are placed in the public domain.

This software is provided 'as-is', without any express or implied warranty.

In no event will the author be held liable for any damages arising from the 

use of the software.

I do hope however that it proves useful to someone.

Other ports

-----------

There have been ports of previous versions of Python to OS/2.

The best known would be that by Jeff Rush, most recently of version 

1.5.2.  Jeff used IBM's Visual Age C++ (v3) for his ports, and his 

patches have been included in the Python 2.6 source distribution.

Andy Zabolotny implemented a port of Python v1.5.2 using the EMX 

development tools.  His patches against the Python v1.5.2 source 

distribution have become the core of this port, and without his efforts 

this port wouldn't exist.  Andy's port also appears to have been 

compiled with his port of gcc 2.95.2 to EMX, which I have but have 

chosen not to use for the binary distribution of this port (see item 16 

of the "YOU HAVE BEEN WARNED" section below).

It is possible to have these earlier ports still usable after installing 

this port - see the README.os2emx.multiple_versions file, contributed by

Dr David Mertz, for a suggested approach to achieving this.

Software requirements

---------------------

This package requires the EMX Runtime package, available from the 

Hobbes (http://hobbes.nmsu.edu/) and LEO (http://archiv.leo.org/) 

archives of OS/2 software.  I have used EMX version 0.9d fix04 in 

developing this port.

My development system is running OS/2 v4 with fixpack 12.

3rd party software which has been linked into dynamically loaded modules:

- ncurses      (see http://dickey.his.com/ for more info, v5.2)

- GNU Readline (Kai Uwe Rommel's port available from Hobbes or LEO, v2.1)

- GNU GDBM     (Kai Uwe Rommel's port available from Hobbes or LEO, v1.7.3)

- zlib         (derived from Hung-Chi Chu's port of v1.1.3, v1.1.4)

- expat        (distributed with Python, v1.95.6)

- GNU UFC      (Kai Uwe Rommel's port available from LEO, v2.0.4)

About this port

---------------

I have attempted to make this port as complete and functional as I can, 

notwithstanding the issues in the "YOU HAVE BEEN WARNED" section below.

Core components:

Python.exe is linked as an a.out executable, ie using EMX method E1 

to compile & link the executable.  This is so that fork() works (see 

"YOU HAVE BEEN WARNED" item 1).

Python26.dll is created as a normal OMF DLL, with an OMF import 

library and module definition file.  There is also an a.out (.a) import 

library to support linking the DLL to a.out executables.  The DLL 

requires the EMX runtime DLLs.

This port has been built with complete support for multithreading.

Modules:

With the exception of modules that have a significant code size, or are 

not recommended or desired for normal use, the standard modules are now 

built into the core DLL rather than configured as dynamically loadable 

modules.  This is for both reasons of performance (startup time) and 

memory use (lots of small DLLs fragment the address space).

I haven't yet changed the building of Python's dynamically loadable 

modules over to using the DistUtils.

See "YOU HAVE BEEN WARNED" item 3 for notes about the fcntl module, and 

"YOU HAVE BEEN WARNED" item 10 for notes about the pwd and grp modules.

This port supports case sensitive module import semantics, matching 

the Windows release.  This can be deactivated by setting the PYTHONCASEOK 

environment variable (the value doesn't matter) - see "YOU HAVE BEEN WARNED" 

item 12.

Optional modules:

Where I've been able to locate the required 3rd party packages already 

ported to OS/2, I've built and included them.

These include ncurses (_curses, _curses_panel), BSD DB (bsddb185), 

GNU GDBM (gdbm, dbm), zlib (zlib), GNU Readline (readline), and GNU UFC 

(crypt).

Expat is now included in the Python release sourceball, and the pyexpat 

module is always built.

I have built these modules statically linked against the 3rd party 

libraries.  Unfortunately my attempts to use the dll version of GNU 

readline have been a dismal failure, in that when the dynamically 

linked readline module is active other modules immediately provoke a 

core dump when imported.

Only the BSD DB package (part of the BSD package distributed with EMX) 

needs source modifications to be used for this port, pertaining to use 

of errno with multithreading.

The other packages, except for ncurses and zlib, needed Makefile changes 

for multithreading support but no source changes.

The _curses_panel module is a potential problem - see "YOU HAVE BEEN 

WARNED" item 13.

Upstream source patches:

No updates to the Python 2.6 release have become available.

Eberhard Mattes' EMXFIX04 update to his EMX 0.9d tools suite includes 

bug fixes for the BSD DB library.  The bsddb module included in this 

port incorporates these fixes.

Library and other distributed Python code:

The Python standard library lives in the Lib directory.  All the standard 

library code included with the Python 2.6 source distribution is included 

in the binary archive, with the exception of the dos-8x3 and tkinter 

subdirectories which have been omitted to reduce the size of the binary 

archive - the dos-8x3 components are unnecessary duplicates and Tkinter 

is not supported by this port (yet).  All the plat-* subdirectories in the 

source distribution have also been omitted, except for the plat-os2emx 

subdirectory.

The Tools and Demo directories contain a collection of Python scripts.  

To reduce the size of the binary archive, the Demo/sgi, Demo/Tix, 

Demo/tkinter, Tools/audiopy and Tools/IDLE subdirectories have been 

omitted as not being supported by this port.  The Misc directory has 

also been omitted.

All subdirectories omitted from the binary archive can be reconstituted 

from the Python 2.6 source distribution, if desired.

Support for building Python extensions:

The Config subdirectory contains the files describing the configuration 

of the interpreter and the Makefile, import libraries for the Python DLL, 

and the module definition file used to create the Python DLL.  The 

Include subdirectory contains all the standard Python header files 

needed for building extensions.

As I don't have the Visual Age C++ compiler, I've made no attempt to 

have this port support extensions built with that compiler.

Packaging

---------

This port is packaged as follows:

- python-2.6-os2emx-bin-03????.zip  (binaries, library modules)

- python-2.6-os2emx-src-03????      (patches+makefiles for non-Python code)

As all the Python specific patches for the port are now part of the 

Python release tarball, only the patches and makefiles involved in 

building external libraries for optional extensions are included in 

the source archive.

Documentation for the Python language, as well as the Python 2.6 

source distibution, can be obtained from the Python website 

(http://www.python.org/) or the Python project pages at Sourceforge 

(http://sf.net/projects/python/).

Installation

------------

Obtain and install, as per the included instructions, the EMX runtime 

package.

Unpack this archive, preserving the subdirectories, in the root directory 

of the drive where you want Python to live.

Add the Python directory (eg C:\Python26) to the PATH and LIBPATH 

variables in CONFIG.SYS.

You should then set the PYTHONHOME and PYTHONPATH environment variables 

in CONFIG.SYS.

PYTHONHOME should be set to Python's top level directory.  PYTHONPATH 

should be set to the semicolon separated list of principal Python library 

directories.

I use:

  SET PYTHONHOME=F:/Python26

  SET PYTHONPATH=F:/Python26/Lib;F:/Python26/Lib/plat-os2emx;

                 F:/Python26/Lib/lib-dynload;F:/Python26/Lib/site-packages

NOTE!:  the PYTHONPATH setting above is linewrapped for this document - it 

should all be on one line in CONFIG.SYS!

If you wish to use the curses module, you should set the TERM and TERMINFO 

environment variables appropriately.

If you don't already have ncurses installed, I have included a copy of the 

EMX subset of the Terminfo database included with the ncurses-5.2 source 

distribution.  This can be used by setting the TERMINFO environment variable 

to the path of the Terminfo subdirectory below the Python home directory.

On my system this looks like:

  SET TERMINFO=F:/Python26/Terminfo

For the TERM environment variable, I would try one of the following:

  SET TERM=ansi

  SET TERM=os2

  SET TERM=window

You will have to reboot your system for these changes to CONFIG.SYS to take 

effect.

If you wish to compile all the included Python library modules to bytecode, 

you can change into the Python home directory and run the COMPILEALL.CMD 

batch file.

You can execute the regression tests included with the Python 2.6 source 

distribution by changing to the Python 2.6 home directory and executing the 

REGRTEST.CMD batch file.  The following tests are known to fail at this 

time:

- test_mhlib (I don't know of any port of MH to OS/2);

- test_strptime (see "YOU HAVE BEEN WARNED" item 22);

- test_time (see "YOU HAVE BEEN WARNED" item 22);

- test_posixpath (see "YOU HAVE BEEN WARNED" item 23).

Note that some of the network related tests expect the loopback interface

(interface "lo", with IP address 127.0.0.1) to be enabled, which from my

experience is not the default configuration.  Additionally, test_popen2

expects the "cat" utility (such as found in ports of the GNU tools) to

be installed.

Building from source

--------------------

With the EMX port now checked into Python's CVS repository, the build 

infrastructure is part of the Python release sourceball.

Prerequisites

First and foremost, you need an operational EMX development installation - 

EMX v0.9d with fix04 (the latest at time of writing) & the gcc 2.8.1 

compiler released by Eberhard Mattes is the recommended setup.

If you have a different version of gcc installed, see "YOU HAVE BEEN 

WARNED" item 16.

Other items of software required:-

- GNU make (I'm using v3.76.1)

- rm, cp, mkdir from the GNU file utilities package

- GNU find

- GNU sed

Procedure

0. all changes mentioned apply to files in the PC/os2emx subdirectory 

   of the Python release source tree.  make is also executed from this 

   directory, so change into this directory before proceeding.

1. decide if you need to change the location of the Python installation.

   If you wish to do this, set the value of the Makefile variable LIB_DIR 

   to the directory you wish to use for PYTHONHOME 

   (eg /usr/local/lib/python2.6).

   If you want Python to find its library without the PYTHONHOME 

   environment variable set, set the value of the Makefile variable 

   FIXED_PYHOME to "yes" (uncomment the appropriate line).

2. If you wish the Python executables (python.exe, pythonpm.exe & pgen.exe) 

   to be installed in a directory other than the PYTHONHOME directory, set 

   the value of the Makefile variable EXE_DIR to the appropriate directory.

3. If you wish the Python core DLL (python26.dll) to be installed in a 

   directory other than the directory in which the Python executables are 

   installed (by default, the PYTHONHOME directory), set the value of the 

   Makefile variable DLL_DIR to the appropriate directory.  This DLL must 

   be placed in a directory on the system's LIBPATH, or that gets set 

   with BEGINLIBPATH or ENDLIBPATH.

4. If you have installed any of the libraries that can be used to build 

   optional Python modules, set the value of the relevant HAVE_<package> 

   Makefile variable to "yes".  The Makefile currently supports:

   library               Makefile variable

   ........................................

   zlib (1.1.4)          HAVE_ZLIB

   GNU UltraFast Crypt   HAVE_UFC

   Tcl/Tk                HAVE_TCLTK (not known to work)

   GNU Readline          HAVE_GREADLINE

   BSD DB (v1.85)        HAVE_BSDDB

   ncurses               HAVE_NCURSES

   GNU gdbm              HAVE_GDBM

   libbz2                HAVE_BZ2

   OpenSSL               HAVE_OPENSSL

   Please note that you need to check that what you have installed 

   is compatible with Python's build options.  In particular, the 

   BSD DB v1.85 library needs to be rebuilt with a source patch for 

   multithread support (doesn't change the library's reentrant status 

   but allows it to be linked to Python which is multithreaded).  

   Widely available binary packages of other librarys & DLLs are 

   not built/linked with multithread support.  Beware!

   Also note that the Makefile currently expects any libraries to be 

   found with the default library search path.  You may need to add 

   -L switches to the LDFLAGS Makefile variable if you have installed 

   libraries in directories not in the default search path (which can 

   be controlled by the LIBRARY_PATH environment variable used by EMX).

5. make

   It is usually a good idea to redirect the stdout and stderr streams 

   of the make process to log files, so that you can review any messages. 

6. make test

   This runs the Python regression tests, and completion is a sign of 

   a usable build.  You should check the list of skipped modules to 

   ensure that any optional modules you selected have been built; 

   checking the list of failures against the list of known failures 

   elsewhere in this document is also prudent.

7. make install

   >>>>>> NOT YET COMPLETE <<<<<< 

8. change to a directory outside the Python source tree and start Python. 

   Check the version and build date to confirm satisfactory installation.

YOU HAVE BEEN WARNED!!

----------------------

I know about a number of nasties in this port.

1.  Eberhard Mattes, author of EMX, writes in his documentation that fork() 

is very inefficient in the OS/2 environment.  It also requires that the 

executable be linked in a.out format rather than OMF.  Use the os.exec 

and/or the os.spawn family of functions where possible.

2.  In the absence of GNU Readline, terminating the interpreter requires a 

control-Z (^Z) followed by a carriage return.  Jeff Rush documented this 

problem in his Python 1.5.2 port.  With Readline, a control-D (^D) works 

as per the standard Unix environment.

3.  EMX only has a partial implementation of fcntl().  The fcntl module 

in this port supports what EMX supports.  If fcntl is important to you, 

please review the EMX C Library Reference (included in .INF format in the 

EMXVIEW.ZIP archive as part of the complete EMX development tools suite).

Because of other side-effects I have modified the test_fcntl.py test 

script to deactivate the exercising of the missing functionality.

4.  the PyBSDDB3 module has been imported into the Python standard

library, with the intent of superceding the BSDDB 1.85 module (bsddb).

As I don't yet have a satisfactory port of Sleepcat's more recent DB

library (3.3.x/4.0.x/4.1.x), I haven't included a binary of this

module.  I have left the Python part of the PyBSDDB package in this

distribution for completeness.

5.  As a consequence of the PyBSDDB3 module being imported, the former 

BSD DB (bsddb) module, linked against the DB v1.85 library from EMX, 

has been renamed bsddb185.  The bsddb185 module will not be built by 

default on most platforms, but in the absence of a PyBSDDB3 module I 

have retained it in the EMX port.

Version 1.85 of the DB library is widely known to have bugs, although 

some patches have become available (and are incorporated into the 

included bsddb185 module).  Unless you have problems with software 

licenses which would rule out GDBM (and the dbm module because it is 

linked against the GDBM library) or need it for file format compatibility, 

you may be better off deleting it and relying on GDBM.

Any code you have which uses the v1.85 bsddb module can be modified to 

use the renamed module by changing

  import bsddb

to

  import bsddb185 as bsddb

6.  The readline module has been linked against ncurses rather than the 

termcap library supplied with EMX.

7.  I have configured this port to use "/" as the preferred path separator 

character, rather than "\" ('\\'), in line with the convention supported 

by EMX.  Backslashes are still supported of course, and still appear in 

unexpected places due to outside sources that don't get normalised.

8.  While the DistUtils components are now functional, other 

packaging/binary handling tools and utilities such as those included in

the Demo and Tools directories - freeze in particular - are unlikely to 

work.  If you do get them going, I'd like to know about your success.

9.  I haven't set out to support the [BEGIN|END]LIBPATH functionality 

supported by one of the earlier ports (Rush's??).  If it works let me know.

10. As a result of the limitations imposed by EMX's library routines, the 

standard extension module pwd only synthesises a simple passwd database, 

and the grp module cannot be supported at all.

I have written pure Python substitutes for pwd and grp, which can process 

real passwd and group files for those applications (such as MailMan) that 

require more than EMX emulates.  I have placed pwd.py and grp.py in 

Lib/plat-os2emx, which is usually before Lib/lib-dynload (which contains 

pwd.pyd) in the PYTHONPATH.  If you have become attached to what pwd.pyd 

supports, you can put Lib/lib-dynload before Lib/plat-os2emx in PYTHONPATH 

or delete/rename pwd.py & grp.py.

pwd.py & grp.py support locating their data files by looking in the 

environment for them in the following sequence:

pwd.py:  $ETC_PASSWD             (%ETC_PASSWD%)

         $ETC/passwd             (%ETC%/passwd)

         $PYTHONHOME/Etc/passwd  (%PYTHONHOME%/Etc/passwd)

grp.py:  $ETC_GROUP              (%ETC_GROUP%)

         $ETC/group              (%ETC%/group)

         $PYTHONHOME/Etc/group   (%PYTHONHOME%/Etc/group)

The ETC_PASSWD and ETC_GROUP environment variables are intended to allow 

support for multiple passwd/grp files, where other applications may not 

support as wide a variety of input variations (drive remappings, 

separators etc).

Both modules support using either the ":" character (Unix standard) or 

";" (OS/2, DOS, Windows standard) field separator character, and pwd.py 

implements the following drive letter conversions for the home_directory and 

shell fields (for the ":" separator only):

         $x  ->  x:

         x;  ->  x:

Example versions of passwd and group are in the Etc subdirectory.  The 

regression tests (test_pwd and test_grp) will fail if valid password and 

group files cannot be found, but should pass otherwise.

Be aware that Python's pwd & group modules are for reading password and 

group information only.

11. EMX's termios routines don't support all of the functionality now 

exposed by the termios module - refer to the EMX documentation to find 

out what is supported.

12. The case sensitive import semantics introduced in Python 2.1 for other 

case insensitive but case preserving file/operating systems (Windows etc), 

have been incorporated into this port, and are active by default.  Setting 

the PYTHONCASEOK environment variable (to any value) reverts to the 

previous (case insensitive) semantics.  This can be an issue with some 

file management utilities that do not preserve the case of file and

directory names.

13. Because I am statically linking ncurses, the _curses_panel 

module has potential problems arising from separate library data areas.

To avoid this, I have configured the _curses_.pyd (imported as 

"_curses_panel") to import the ncurses symbols it needs from _curses.dll 

(which is the curses module, but with a .dll extension rather than .pyd 

so that the dynamic loader can actually import the symbols from it as a 

DLL).

The site module (Lib/site.py) has code added to tweak BEGINLIBPATH so

that _curses.dll is found when _curses_panel is imported.  If you have

problems attempting to use the _curses_panel support please let me know,

and I'll have another look at this.

14. sys.platform reports "os2emx" instead of "os2".  os.name still 

reports "os2".  This change was to make it easier to distinguish between 

the VAC++ build (formerly maintained by Michael Muller) and the EMX build 

(this port), principally for DistUtils.

15. it appears that the %W substitution in the EMX strftime() routine has 

an off-by-one bug.  strftime was listed as passing the regression tests 

in previous releases, but this fact appears to have been an oversight in 

the regression test suite.  To fix this really requires a portable 

strftime routine - I'm looking into using one from FreeBSD, but its not 

ready yet.

16. I have successfully built this port with Andy Zabolotny's ports of 

pgcc 2.95 and gcc 3.2.1, in addition to EM's gcc 2.8.1.  To use the 

bsddb185 module with the gcc 3.2.1 build, I had to recompile the DB library 

with gcc 3.2.1 - I don't know why, but trying to import the module built 

against a DB library compiled with gcc 2.8.1 would result in a SYS3175 

error.

I have not attempted to compile Python with any version of gcc prior to 

v2.8.1.

This release sees the default optimisation change to 

"-O3 -fomit-frame-pointer -mprobe".  This works fine too for pgcc 2.95 

but not for gcc 3.2.1.

With gcc 3.2.1, -O3 causes 2 unexpected test failures: test_format and 

test_unicode.  Both these tests pass if -O2 is instead of -O3 with this