1 # subprocess - Subprocesses with accessible I/O streams

     2 #

     3 # For more information about this module, see PEP 324.

     4 #

     5 # This module should remain compatible with Python 2.2, see PEP 291.

     6 #

     7 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>

     8 #

     9 # Licensed to PSF under a Contributor Agreement.

    10 # See http://www.python.org/2.4/license for licensing details.

    11 

    12 r"""subprocess - Subprocesses with accessible I/O streams

    13 

    14 This module allows you to spawn processes, connect to their

    15 input/output/error pipes, and obtain their return codes.  This module

    16 intends to replace several other, older modules and functions, like:

    17 

    18 os.system

    19 os.spawn*

    20 os.popen*

    21 popen2.*

    22 commands.*

    23 

    24 Information about how the subprocess module can be used to replace these

    25 modules and functions can be found below.

    26 

    27 

    28 

    29 Using the subprocess module

    30 ===========================

    31 This module defines one class called Popen:

    32 

    33 class Popen(args, bufsize=0, executable=None,

    34             stdin=None, stdout=None, stderr=None,

    35             preexec_fn=None, close_fds=False, shell=False,

    36             cwd=None, env=None, universal_newlines=False,

    37             startupinfo=None, creationflags=0):

    38 

    39 

    40 Arguments are:

    41 

    42 args should be a string, or a sequence of program arguments.  The

    43 program to execute is normally the first item in the args sequence or

    44 string, but can be explicitly set by using the executable argument.

    45 

    46 On UNIX, with shell=False (default): In this case, the Popen class

    47 uses os.execvp() to execute the child program.  args should normally

    48 be a sequence.  A string will be treated as a sequence with the string

    49 as the only item (the program to execute).

    50 

    51 On UNIX, with shell=True: If args is a string, it specifies the

    52 command string to execute through the shell.  If args is a sequence,

    53 the first item specifies the command string, and any additional items

    54 will be treated as additional shell arguments.

    55 

    56 On Windows: the Popen class uses CreateProcess() to execute the child

    57 program, which operates on strings.  If args is a sequence, it will be

    58 converted to a string using the list2cmdline method.  Please note that

    59 not all MS Windows applications interpret the command line the same

    60 way: The list2cmdline is designed for applications using the same

    61 rules as the MS C runtime.

    62 

    63 bufsize, if given, has the same meaning as the corresponding argument

    64 to the built-in open() function: 0 means unbuffered, 1 means line

    65 buffered, any other positive value means use a buffer of

    66 (approximately) that size.  A negative bufsize means to use the system

    67 default, which usually means fully buffered.  The default value for

    68 bufsize is 0 (unbuffered).

    69 

    70 stdin, stdout and stderr specify the executed programs' standard

    71 input, standard output and standard error file handles, respectively.

    72 Valid values are PIPE, an existing file descriptor (a positive

    73 integer), an existing file object, and None.  PIPE indicates that a

    74 new pipe to the child should be created.  With None, no redirection

    75 will occur; the child's file handles will be inherited from the

    76 parent.  Additionally, stderr can be STDOUT, which indicates that the

    77 stderr data from the applications should be captured into the same

    78 file handle as for stdout.

    79 

    80 If preexec_fn is set to a callable object, this object will be called

    81 in the child process just before the child is executed.

    82 

    83 If close_fds is true, all file descriptors except 0, 1 and 2 will be

    84 closed before the child process is executed.

    85 

    86 if shell is true, the specified command will be executed through the

    87 shell.

    88 

    89 If cwd is not None, the current directory will be changed to cwd

    90 before the child is executed.

    91 

    92 If env is not None, it defines the environment variables for the new

    93 process.

    94 

    95 If universal_newlines is true, the file objects stdout and stderr are

    96 opened as a text files, but lines may be terminated by any of '\n',

    97 the Unix end-of-line convention, '\r', the Macintosh convention or

    98 '\r\n', the Windows convention.  All of these external representations

    99 are seen as '\n' by the Python program.  Note: This feature is only

   100 available if Python is built with universal newline support (the

   101 default).  Also, the newlines attribute of the file objects stdout,

   102 stdin and stderr are not updated by the communicate() method.

   103 

   104 The startupinfo and creationflags, if given, will be passed to the

   105 underlying CreateProcess() function.  They can specify things such as

   106 appearance of the main window and priority for the new process.

   107 (Windows only)

   108 

   109 

   110 This module also defines two shortcut functions:

   111 

   112 call(*popenargs, **kwargs):

   113     Run command with arguments.  Wait for command to complete, then

   114     return the returncode attribute.

   115 

   116     The arguments are the same as for the Popen constructor.  Example:

   117 

   118     retcode = call(["ls", "-l"])

   119 

   120 check_call(*popenargs, **kwargs):

   121     Run command with arguments.  Wait for command to complete.  If the

   122     exit code was zero then return, otherwise raise

   123     CalledProcessError.  The CalledProcessError object will have the

   124     return code in the returncode attribute.

   125 

   126     The arguments are the same as for the Popen constructor.  Example:

   127 

   128     check_call(["ls", "-l"])

   129 

   130 Exceptions

   131 ----------

   132 Exceptions raised in the child process, before the new program has

   133 started to execute, will be re-raised in the parent.  Additionally,

   134 the exception object will have one extra attribute called

   135 'child_traceback', which is a string containing traceback information

   136 from the childs point of view.

   137 

   138 The most common exception raised is OSError.  This occurs, for

   139 example, when trying to execute a non-existent file.  Applications

   140 should prepare for OSErrors.

   141 

   142 A ValueError will be raised if Popen is called with invalid arguments.

   143 

   144 check_call() will raise CalledProcessError, if the called process

   145 returns a non-zero return code.

   146 

   147 

   148 Security

   149 --------

   150 Unlike some other popen functions, this implementation will never call

   151 /bin/sh implicitly.  This means that all characters, including shell

   152 metacharacters, can safely be passed to child processes.

   153 

   154 

   155 Popen objects

   156 =============

   157 Instances of the Popen class have the following methods:

   158 

   159 poll()

   160     Check if child process has terminated.  Returns returncode

   161     attribute.

   162 

   163 wait()

   164     Wait for child process to terminate.  Returns returncode attribute.

   165 

   166 communicate(input=None)

   167     Interact with process: Send data to stdin.  Read data from stdout

   168     and stderr, until end-of-file is reached.  Wait for process to

   169     terminate.  The optional input argument should be a string to be

   170     sent to the child process, or None, if no data should be sent to

   171     the child.

   172 

   173     communicate() returns a tuple (stdout, stderr).

   174 

   175     Note: The data read is buffered in memory, so do not use this

   176     method if the data size is large or unlimited.

   177 

   178 The following attributes are also available:

   179 

   180 stdin

   181     If the stdin argument is PIPE, this attribute is a file object

   182     that provides input to the child process.  Otherwise, it is None.

   183 

   184 stdout

   185     If the stdout argument is PIPE, this attribute is a file object

   186     that provides output from the child process.  Otherwise, it is

   187     None.

   188 

   189 stderr

   190     If the stderr argument is PIPE, this attribute is file object that

   191     provides error output from the child process.  Otherwise, it is

   192     None.

   193 

   194 pid

   195     The process ID of the child process.

   196 

   197 returncode

   198     The child return code.  A None value indicates that the process

   199     hasn't terminated yet.  A negative value -N indicates that the

   200     child was terminated by signal N (UNIX only).

   201 

   202 

   203 Replacing older functions with the subprocess module

   204 ====================================================

   205 In this section, "a ==> b" means that b can be used as a replacement

   206 for a.

   207 

   208 Note: All functions in this section fail (more or less) silently if

   209 the executed program cannot be found; this module raises an OSError

   210 exception.

   211 

   212 In the following examples, we assume that the subprocess module is

   213 imported with "from subprocess import *".

   214 

   215 

   216 Replacing /bin/sh shell backquote

   217 ---------------------------------

   218 output=`mycmd myarg`

   219 ==>

   220 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]

   221 

   222 

   223 Replacing shell pipe line

   224 -------------------------

   225 output=`dmesg | grep hda`

   226 ==>

   227 p1 = Popen(["dmesg"], stdout=PIPE)

   228 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)

   229 output = p2.communicate()[0]

   230 

   231 

   232 Replacing os.system()

   233 ---------------------

   234 sts = os.system("mycmd" + " myarg")

   235 ==>

   236 p = Popen("mycmd" + " myarg", shell=True)

   237 pid, sts = os.waitpid(p.pid, 0)

   238 

   239 Note:

   240 

   241 * Calling the program through the shell is usually not required.

   242 

   243 * It's easier to look at the returncode attribute than the

   244   exitstatus.

   245 

   246 A more real-world example would look like this:

   247 

   248 try:

   249     retcode = call("mycmd" + " myarg", shell=True)

   250     if retcode < 0:

   251         print >>sys.stderr, "Child was terminated by signal", -retcode

   252     else:

   253         print >>sys.stderr, "Child returned", retcode

   254 except OSError, e:

   255     print >>sys.stderr, "Execution failed:", e

   256 

   257 

   258 Replacing os.spawn*

   259 -------------------

   260 P_NOWAIT example:

   261 

   262 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")

   263 ==>

   264 pid = Popen(["/bin/mycmd", "myarg"]).pid

   265 

   266 

   267 P_WAIT example:

   268 

   269 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")

   270 ==>

   271 retcode = call(["/bin/mycmd", "myarg"])

   272 

   273 

   274 Vector example:

   275 

   276 os.spawnvp(os.P_NOWAIT, path, args)

   277 ==>

   278 Popen([path] + args[1:])

   279 

   280 

   281 Environment example:

   282 

   283 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)

   284 ==>

   285 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})

   286 

   287 

   288 Replacing os.popen*

   289 -------------------

   290 pipe = os.popen(cmd, mode='r', bufsize)

   291 ==>

   292 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout

   293 

   294 pipe = os.popen(cmd, mode='w', bufsize)

   295 ==>

   296 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin

   297 

   298 

   299 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)

   300 ==>

   301 p = Popen(cmd, shell=True, bufsize=bufsize,

   302           stdin=PIPE, stdout=PIPE, close_fds=True)

   303 (child_stdin, child_stdout) = (p.stdin, p.stdout)

   304 

   305 

   306 (child_stdin,

   307  child_stdout,

   308  child_stderr) = os.popen3(cmd, mode, bufsize)

   309 ==>

   310 p = Popen(cmd, shell=True, bufsize=bufsize,

   311           stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)

   312 (child_stdin,

   313  child_stdout,

   314  child_stderr) = (p.stdin, p.stdout, p.stderr)

   315 

   316 

   317 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)

   318 ==>

   319 p = Popen(cmd, shell=True, bufsize=bufsize,

   320           stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)

   321 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)

   322 

   323 

   324 Replacing popen2.*

   325 ------------------

   326 Note: If the cmd argument to popen2 functions is a string, the command

   327 is executed through /bin/sh.  If it is a list, the command is directly

   328 executed.

   329 

   330 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)

   331 ==>

   332 p = Popen(["somestring"], shell=True, bufsize=bufsize

   333           stdin=PIPE, stdout=PIPE, close_fds=True)

   334 (child_stdout, child_stdin) = (p.stdout, p.stdin)

   335 

   336 

   337 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)

   338 ==>

   339 p = Popen(["mycmd", "myarg"], bufsize=bufsize,

   340           stdin=PIPE, stdout=PIPE, close_fds=True)

   341 (child_stdout, child_stdin) = (p.stdout, p.stdin)

   342 

   343 The popen2.Popen3 and popen2.Popen4 basically works as subprocess.Popen,

   344 except that:

   345 

   346 * subprocess.Popen raises an exception if the execution fails

   347 * the capturestderr argument is replaced with the stderr argument.

   348 * stdin=PIPE and stdout=PIPE must be specified.

   349 * popen2 closes all filedescriptors by default, but you have to specify

   350   close_fds=True with subprocess.Popen.

   351 

   352 

   353 """

   354 

   355 import sys

   356 mswindows = (sys.platform == "win32")

   357 

   358 import os

   359 import types

   360 import traceback

   361 import gc

   362 

   363 # Exception classes used by this module.

   364 class CalledProcessError(Exception):

   365     """This exception is raised when a process run by check_call() returns

   366     a non-zero exit status.  The exit status will be stored in the

   367     returncode attribute."""

   368     def __init__(self, returncode, cmd):

   369         self.returncode = returncode

   370         self.cmd = cmd

   371     def __str__(self):

   372         return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)

   373 

   374 

   375 if mswindows:

   376     import threading

   377     import msvcrt

   378     if 0: # <-- change this to use pywin32 instead of the _subprocess driver

   379         import pywintypes

   380         from win32api import GetStdHandle, STD_INPUT_HANDLE, \

   381                              STD_OUTPUT_HANDLE, STD_ERROR_HANDLE

   382         from win32api import GetCurrentProcess, DuplicateHandle, \

   383                              GetModuleFileName, GetVersion

   384         from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE

   385         from win32pipe import CreatePipe

   386         from win32process import CreateProcess, STARTUPINFO, \

   387                                  GetExitCodeProcess, STARTF_USESTDHANDLES, \

   388                                  STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE

   389         from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0

   390     else:

   391         from _subprocess import *

   392         class STARTUPINFO:

   393             dwFlags = 0

   394             hStdInput = None

   395             hStdOutput = None

   396             hStdError = None

   397             wShowWindow = 0

   398         class pywintypes:

   399             error = IOError

   400 else:

   401     import select

   402     import errno

   403     import fcntl

   404     import pickle

   405 

   406 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"]

   407 

   408 try:

   409     MAXFD = os.sysconf("SC_OPEN_MAX")

   410 except:

   411     MAXFD = 256

   412 

   413 # True/False does not exist on 2.2.0

   414 try:

   415     False

   416 except NameError:

   417     False = 0

   418     True = 1

   419 

   420 _active = []

   421 

   422 def _cleanup():

   423     for inst in _active[:]:

   424         if inst.poll(_deadstate=sys.maxint) >= 0:

   425             try:

   426                 _active.remove(inst)

   427             except ValueError:

   428                 # This can happen if two threads create a new Popen instance.

   429                 # It's harmless that it was already removed, so ignore.

   430                 pass

   431 

   432 PIPE = -1

   433 STDOUT = -2

   434 

   435 

   436 def call(*popenargs, **kwargs):

   437     """Run command with arguments.  Wait for command to complete, then

   438     return the returncode attribute.

   439 

   440     The arguments are the same as for the Popen constructor.  Example:

   441 

   442     retcode = call(["ls", "-l"])

   443     """

   444     return Popen(*popenargs, **kwargs).wait()

   445 

   446 

   447 def check_call(*popenargs, **kwargs):

   448     """Run command with arguments.  Wait for command to complete.  If

   449     the exit code was zero then return, otherwise raise

   450     CalledProcessError.  The CalledProcessError object will have the

   451     return code in the returncode attribute.

   452 

   453     The arguments are the same as for the Popen constructor.  Example:

   454 

   455     check_call(["ls", "-l"])

   456     """

   457     retcode = call(*popenargs, **kwargs)

   458     cmd = kwargs.get("args")

   459     if cmd is None:

   460         cmd = popenargs[0]

   461     if retcode:

   462         raise CalledProcessError(retcode, cmd)

   463     return retcode

   464 

   465 

   466 def list2cmdline(seq):

   467     """

   468     Translate a sequence of arguments into a command line

   469     string, using the same rules as the MS C runtime:

   470 

   471     1) Arguments are delimited by white space, which is either a

   472        space or a tab.

   473 

   474     2) A string surrounded by double quotation marks is

   475        interpreted as a single argument, regardless of white space

   476        contained within.  A quoted string can be embedded in an

   477        argument.

   478 

   479     3) A double quotation mark preceded by a backslash is

   480        interpreted as a literal double quotation mark.

   481 

   482     4) Backslashes are interpreted literally, unless they

   483        immediately precede a double quotation mark.

   484 

   485     5) If backslashes immediately precede a double quotation mark,

   486        every pair of backslashes is interpreted as a literal

   487        backslash.  If the number of backslashes is odd, the last

   488        backslash escapes the next double quotation mark as

   489        described in rule 3.

   490     """

   491 

   492     # See

   493     # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp

   494     result = []

   495     needquote = False

   496     for arg in seq:

   497         bs_buf = []

   498 

   499         # Add a space to separate this argument from the others

   500         if result:

   501             result.append(' ')

   502 

   503         needquote = (" " in arg) or ("\t" in arg) or arg == ""

   504         if needquote:

   505             result.append('"')

   506 

   507         for c in arg:

   508             if c == '\\':

   509                 # Don't know if we need to double yet.

   510                 bs_buf.append(c)

   511             elif c == '"':

   512                 # Double backspaces.

   513                 result.append('\\' * len(bs_buf)*2)

   514                 bs_buf = []

   515                 result.append('\\"')

   516             else:

   517                 # Normal char

   518                 if bs_buf:

   519                     result.extend(bs_buf)

   520                     bs_buf = []

   521                 result.append(c)

   522 

   523         # Add remaining backspaces, if any.

   524         if bs_buf:

   525             result.extend(bs_buf)

   526 

   527         if needquote:

   528             result.extend(bs_buf)

   529             result.append('"')

   530 

   531     return ''.join(result)

   532 

   533 

   534 class Popen(object):

   535     def __init__(self, args, bufsize=0, executable=None,

   536                  stdin=None, stdout=None, stderr=None,

   537                  preexec_fn=None, close_fds=False, shell=False,

   538                  cwd=None, env=None, universal_newlines=False,

   539                  startupinfo=None, creationflags=0):

   540         """Create new Popen instance."""

   541         _cleanup()

   542 

   543         self._child_created = False

   544         if not isinstance(bufsize, (int, long)):

   545             raise TypeError("bufsize must be an integer")

   546 

   547         if mswindows:

   548             if preexec_fn is not None:

   549                 raise ValueError("preexec_fn is not supported on Windows "

   550                                  "platforms")

   551             if close_fds:

   552                 raise ValueError("close_fds is not supported on Windows "

   553                                  "platforms")

   554         else:

   555             # POSIX

   556             if startupinfo is not None:

   557                 raise ValueError("startupinfo is only supported on Windows "

   558                                  "platforms")

   559             if creationflags != 0:

   560                 raise ValueError("creationflags is only supported on Windows "

   561                                  "platforms")

   562 

   563         self.stdin = None

   564         self.stdout = None

   565         self.stderr = None

   566         self.pid = None

   567         self.returncode = None

   568         self.universal_newlines = universal_newlines

   569 

   570         # Input and output objects. The general principle is like

   571         # this:

   572         #

   573         # Parent                   Child

   574         # ------                   -----

   575         # p2cwrite   ---stdin--->  p2cread

   576         # c2pread    <--stdout---  c2pwrite

   577         # errread    <--stderr---  errwrite

   578         #

   579         # On POSIX, the child objects are file descriptors.  On

   580         # Windows, these are Windows file handles.  The parent objects

   581         # are file descriptors on both platforms.  The parent objects

   582         # are None when not using PIPEs. The child objects are None

   583         # when not redirecting.

   584 

   585         (p2cread, p2cwrite,

   586          c2pread, c2pwrite,

   587          errread, errwrite) = self._get_handles(stdin, stdout, stderr)

   588 

   589         self._execute_child(args, executable, preexec_fn, close_fds,

   590                             cwd, env, universal_newlines,

   591                             startupinfo, creationflags, shell,

   592                             p2cread, p2cwrite,

   593                             c2pread, c2pwrite,

   594                             errread, errwrite)

   595 

   596         # On Windows, you cannot just redirect one or two handles: You

   597         # either have to redirect all three or none. If the subprocess

   598         # user has only redirected one or two handles, we are

   599         # automatically creating PIPEs for the rest. We should close

   600         # these after the process is started. See bug #1124861.

   601         if mswindows:

   602             if stdin is None and p2cwrite is not None:

   603                 os.close(p2cwrite)

   604                 p2cwrite = None

   605             if stdout is None and c2pread is not None:

   606                 os.close(c2pread)

   607                 c2pread = None

   608             if stderr is None and errread is not None:

   609                 os.close(errread)

   610                 errread = None

   611 

   612         if p2cwrite:

   613             self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)

   614         if c2pread:

   615             if universal_newlines:

   616                 self.stdout = os.fdopen(c2pread, 'rU', bufsize)

   617             else:

   618                 self.stdout = os.fdopen(c2pread, 'rb', bufsize)

   619         if errread:

   620             if universal_newlines:

   621                 self.stderr = os.fdopen(errread, 'rU', bufsize)

   622             else:

   623                 self.stderr = os.fdopen(errread, 'rb', bufsize)

   624 

   625 

   626     def _translate_newlines(self, data):

   627         data = data.replace("\r\n", "\n")

   628         data = data.replace("\r", "\n")

   629         return data

   630 

   631 

   632     def __del__(self, sys=sys):

   633         if not self._child_created:

   634             # We didn't get to successfully create a child process.

   635             return

   636         # In case the child hasn't been waited on, check if it's done.

   637         self.poll(_deadstate=sys.maxint)

   638         if self.returncode is None and _active is not None:

   639             # Child is still running, keep us alive until we can wait on it.

   640             _active.append(self)

   641 

   642 

   643     def communicate(self, input=None):

   644         """Interact with process: Send data to stdin.  Read data from

   645         stdout and stderr, until end-of-file is reached.  Wait for

   646         process to terminate.  The optional input argument should be a

   647         string to be sent to the child process, or None, if no data

   648         should be sent to the child.

   649 

   650         communicate() returns a tuple (stdout, stderr)."""

   651 

   652         # Optimization: If we are only using one pipe, or no pipe at

   653         # all, using select() or threads is unnecessary.

   654         if [self.stdin, self.stdout, self.stderr].count(None) >= 2:

   655             stdout = None

   656             stderr = None

   657             if self.stdin:

   658                 if input:

   659                     self.stdin.write(input)

   660                 self.stdin.close()

   661             elif self.stdout:

   662                 stdout = self.stdout.read()

   663             elif self.stderr:

   664                 stderr = self.stderr.read()

   665             self.wait()

   666             return (stdout, stderr)

   667 

   668         return self._communicate(input)

   669 

   670 

   671     if mswindows:

   672         #

   673         # Windows methods

   674         #

   675         def _get_handles(self, stdin, stdout, stderr):

   676             """Construct and return tupel with IO objects:

   677             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite

   678             """

   679             if stdin is None and stdout is None and stderr is None:

   680                 return (None, None, None, None, None, None)

   681 

   682             p2cread, p2cwrite = None, None

   683             c2pread, c2pwrite = None, None

   684             errread, errwrite = None, None

   685 

   686             if stdin is None:

   687                 p2cread = GetStdHandle(STD_INPUT_HANDLE)

   688             if p2cread is not None:

   689                 pass

   690             elif stdin is None or stdin == PIPE:

   691                 p2cread, p2cwrite = CreatePipe(None, 0)

   692                 # Detach and turn into fd

   693                 p2cwrite = p2cwrite.Detach()

   694                 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)

   695             elif isinstance(stdin, int):

   696                 p2cread = msvcrt.get_osfhandle(stdin)

   697             else:

   698                 # Assuming file-like object

   699                 p2cread = msvcrt.get_osfhandle(stdin.fileno())

   700             p2cread = self._make_inheritable(p2cread)

   701 

   702             if stdout is None:

   703                 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)

   704             if c2pwrite is not None:

   705                 pass

   706             elif stdout is None or stdout == PIPE:

   707                 c2pread, c2pwrite = CreatePipe(None, 0)

   708                 # Detach and turn into fd

   709                 c2pread = c2pread.Detach()

   710                 c2pread = msvcrt.open_osfhandle(c2pread, 0)

   711             elif isinstance(stdout, int):

   712                 c2pwrite = msvcrt.get_osfhandle(stdout)

   713             else:

   714                 # Assuming file-like object

   715                 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())

   716             c2pwrite = self._make_inheritable(c2pwrite)

   717 

   718             if stderr is None:

   719                 errwrite = GetStdHandle(STD_ERROR_HANDLE)

   720             if errwrite is not None:

   721                 pass

   722             elif stderr is None or stderr == PIPE:

   723                 errread, errwrite = CreatePipe(None, 0)

   724                 # Detach and turn into fd

   725                 errread = errread.Detach()

   726                 errread = msvcrt.open_osfhandle(errread, 0)

   727             elif stderr == STDOUT:

   728                 errwrite = c2pwrite

   729             elif isinstance(stderr, int):

   730                 errwrite = msvcrt.get_osfhandle(stderr)

   731             else:

   732                 # Assuming file-like object

   733                 errwrite = msvcrt.get_osfhandle(stderr.fileno())

   734             errwrite = self._make_inheritable(errwrite)

   735 

   736             return (p2cread, p2cwrite,

   737                     c2pread, c2pwrite,

   738                     errread, errwrite)

   739 

   740 

   741         def _make_inheritable(self, handle):

   742             """Return a duplicate of handle, which is inheritable"""

   743             return DuplicateHandle(GetCurrentProcess(), handle,

   744                                    GetCurrentProcess(), 0, 1,

   745                                    DUPLICATE_SAME_ACCESS)

   746 

   747 

   748         def _find_w9xpopen(self):

   749             """Find and return absolut path to w9xpopen.exe"""

   750             w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),

   751                                     "w9xpopen.exe")

   752             if not os.path.exists(w9xpopen):

   753                 # Eeek - file-not-found - possibly an embedding

   754                 # situation - see if we can locate it in sys.exec_prefix

   755                 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),

   756                                         "w9xpopen.exe")

   757                 if not os.path.exists(w9xpopen):

   758                     raise RuntimeError("Cannot locate w9xpopen.exe, which is "

   759                                        "needed for Popen to work with your "

   760                                        "shell or platform.")

   761             return w9xpopen

   762 

   763 

   764         def _execute_child(self, args, executable, preexec_fn, close_fds,

   765                            cwd, env, universal_newlines,

   766                            startupinfo, creationflags, shell,

   767                            p2cread, p2cwrite,

   768                            c2pread, c2pwrite,

   769                            errread, errwrite):

   770             """Execute program (MS Windows version)"""

   771 

   772             if not isinstance(args, types.StringTypes):

   773                 args = list2cmdline(args)

   774 

   775             # Process startup details

   776             if startupinfo is None:

   777                 startupinfo = STARTUPINFO()

   778             if None not in (p2cread, c2pwrite, errwrite):

   779                 startupinfo.dwFlags |= STARTF_USESTDHANDLES

   780                 startupinfo.hStdInput = p2cread

   781                 startupinfo.hStdOutput = c2pwrite

   782                 startupinfo.hStdError = errwrite

   783 

   784             if shell:

   785                 startupinfo.dwFlags |= STARTF_USESHOWWINDOW

   786                 startupinfo.wShowWindow = SW_HIDE

   787                 comspec = os.environ.get("COMSPEC", "cmd.exe")

   788                 args = comspec + " /c " + args

   789                 if (GetVersion() >= 0x80000000L or

   790                         os.path.basename(comspec).lower() == "command.com"):

   791                     # Win9x, or using command.com on NT. We need to

   792                     # use the w9xpopen intermediate program. For more

   793                     # information, see KB Q150956

   794                     # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)

   795                     w9xpopen = self._find_w9xpopen()

   796                     args = '"%s" %s' % (w9xpopen, args)

   797                     # Not passing CREATE_NEW_CONSOLE has been known to

   798                     # cause random failures on win9x.  Specifically a

   799                     # dialog: "Your program accessed mem currently in

   800                     # use at xxx" and a hopeful warning about the

   801                     # stability of your system.  Cost is Ctrl+C wont

   802                     # kill children.

   803                     creationflags |= CREATE_NEW_CONSOLE

   804 

   805             # Start the process

   806             try:

   807                 hp, ht, pid, tid = CreateProcess(executable, args,

   808                                          # no special security

   809                                          None, None,

   810                                          # must inherit handles to pass std

   811                                          # handles

   812                                          1,

   813                                          creationflags,

   814                                          env,

   815                                          cwd,

   816                                          startupinfo)

   817             except pywintypes.error, e:

   818                 # Translate pywintypes.error to WindowsError, which is

   819                 # a subclass of OSError.  FIXME: We should really

   820                 # translate errno using _sys_errlist (or simliar), but

   821                 # how can this be done from Python?

   822                 raise WindowsError(*e.args)

   823 

   824             # Retain the process handle, but close the thread handle

   825             self._child_created = True

   826             self._handle = hp

   827             self.pid = pid

   828             ht.Close()

   829 

   830             # Child is launched. Close the parent's copy of those pipe

   831             # handles that only the child should have open.  You need

   832             # to make sure that no handles to the write end of the

   833             # output pipe are maintained in this process or else the

   834             # pipe will not close when the child process exits and the

   835             # ReadFile will hang.

   836             if p2cread is not None:

   837                 p2cread.Close()

   838             if c2pwrite is not None:

   839                 c2pwrite.Close()

   840             if errwrite is not None:

   841                 errwrite.Close()

   842 

   843 

   844         def poll(self, _deadstate=None):

   845             """Check if child process has terminated.  Returns returncode

   846             attribute."""

   847             if self.returncode is None:

   848                 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:

   849                     self.returncode = GetExitCodeProcess(self._handle)

   850             return self.returncode

   851 

   852 

   853         def wait(self):

   854             """Wait for child process to terminate.  Returns returncode

   855             attribute."""

   856             if self.returncode is None:

   857                 obj = WaitForSingleObject(self._handle, INFINITE)

   858                 self.returncode = GetExitCodeProcess(self._handle)

   859             return self.returncode

   860 

   861 

   862         def _readerthread(self, fh, buffer):

   863             buffer.append(fh.read())

   864 

   865 

   866         def _communicate(self, input):

   867             stdout = None # Return

   868             stderr = None # Return

   869 

   870             if self.stdout:

   871                 stdout = []

   872                 stdout_thread = threading.Thread(target=self._readerthread,

   873                                                  args=(self.stdout, stdout))

   874                 stdout_thread.setDaemon(True)

   875                 stdout_thread.start()

   876             if self.stderr:

   877                 stderr = []

   878                 stderr_thread = threading.Thread(target=self._readerthread,

   879                                                  args=(self.stderr, stderr))

   880                 stderr_thread.setDaemon(True)

   881                 stderr_thread.start()

   882 

   883             if self.stdin:

   884                 if input is not None:

   885                     self.stdin.write(input)

   886                 self.stdin.close()

   887 

   888             if self.stdout:

   889                 stdout_thread.join()

   890             if self.stderr:

   891                 stderr_thread.join()

   892 

   893             # All data exchanged.  Translate lists into strings.

   894             if stdout is not None:

   895                 stdout = stdout[0]

   896             if stderr is not None:

   897                 stderr = stderr[0]

   898 

   899             # Translate newlines, if requested.  We cannot let the file

   900             # object do the translation: It is based on stdio, which is

   901             # impossible to combine with select (unless forcing no

   902             # buffering).

   903             if self.universal_newlines and hasattr(file, 'newlines'):

   904                 if stdout:

   905                     stdout = self._translate_newlines(stdout)

   906                 if stderr:

   907                     stderr = self._translate_newlines(stderr)

   908 

   909             self.wait()

   910             return (stdout, stderr)

   911 

   912     else:

   913         #

   914         # POSIX methods

   915         #

   916         def _get_handles(self, stdin, stdout, stderr):

   917             """Construct and return tupel with IO objects:

   918             p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite

   919             """

   920             p2cread, p2cwrite = None, None

   921             c2pread, c2pwrite = None, None

   922             errread, errwrite = None, None

   923 

   924             if stdin is None:

   925                 pass

   926             elif stdin == PIPE:

   927                 p2cread, p2cwrite = os.pipe()

   928             elif isinstance(stdin, int):

   929                 p2cread = stdin

   930             else:

   931                 # Assuming file-like object

   932                 p2cread = stdin.fileno()

   933 

   934             if stdout is None:

   935                 pass

   936             elif stdout == PIPE:

   937                 c2pread, c2pwrite = os.pipe()

   938             elif isinstance(stdout, int):

   939                 c2pwrite = stdout

   940             else:

   941                 # Assuming file-like object

   942                 c2pwrite = stdout.fileno()

   943 

   944             if stderr is None:

   945                 pass

   946             elif stderr == PIPE:

   947                 errread, errwrite = os.pipe()

   948             elif stderr == STDOUT:

   949                 errwrite = c2pwrite

   950             elif isinstance(stderr, int):

   951                 errwrite = stderr

   952             else:

   953                 # Assuming file-like object

   954                 errwrite = stderr.fileno()

   955 

   956             return (p2cread, p2cwrite,

   957                     c2pread, c2pwrite,

   958                     errread, errwrite)

   959 

   960 

   961         def _set_cloexec_flag(self, fd):

   962             try:

   963                 cloexec_flag = fcntl.FD_CLOEXEC

   964             except AttributeError:

   965                 cloexec_flag = 1

   966 

   967             old = fcntl.fcntl(fd, fcntl.F_GETFD)

   968             fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)

   969 

   970 

   971         def _close_fds(self, but):

   972             for i in xrange(3, MAXFD):

   973                 if i == but:

   974                     continue

   975                 try:

   976                     os.close(i)

   977                 except:

   978                     pass

   979 

   980 

   981         def _execute_child(self, args, executable, preexec_fn, close_fds,

   982                            cwd, env, universal_newlines,

   983                            startupinfo, creationflags, shell,

   984                            p2cread, p2cwrite,

   985                            c2pread, c2pwrite,

   986                            errread, errwrite):

   987             """Execute program (POSIX version)"""

   988 

   989             if isinstance(args, types.StringTypes):

   990                 args = [args]

   991             else:

   992                 args = list(args)

   993 

   994             if shell:

   995                 args = ["/bin/sh", "-c"] + args

   996 

   997             if executable is None:

   998                 executable = args[0]

   999 

  1000             # For transferring possible exec failure from child to parent

  1001             # The first char specifies the exception type: 0 means

  1002             # OSError, 1 means some other error.

  1003             errpipe_read, errpipe_write = os.pipe()

  1004             self._set_cloexec_flag(errpipe_write)

  1005 

  1006             gc_was_enabled = gc.isenabled()

  1007             # Disable gc to avoid bug where gc -> file_dealloc ->

  1008             # write to stderr -> hang.  http://bugs.python.org/issue1336

  1009             gc.disable()

  1010             try:

  1011                 self.pid = os.fork()

  1012             except:

  1013                 if gc_was_enabled:

  1014                     gc.enable()

  1015                 raise

  1016             self._child_created = True

  1017             if self.pid == 0:

  1018                 # Child

  1019                 try:

  1020                     # Close parent's pipe ends

  1021                     if p2cwrite:

  1022                         os.close(p2cwrite)

  1023                     if c2pread:

  1024                         os.close(c2pread)

  1025                     if errread:

  1026                         os.close(errread)

  1027                     os.close(errpipe_read)

  1028 

  1029                     # Dup fds for child

  1030                     if p2cread:

  1031                         os.dup2(p2cread, 0)

  1032                     if c2pwrite:

  1033                         os.dup2(c2pwrite, 1)

  1034                     if errwrite:

  1035                         os.dup2(errwrite, 2)

  1036 

  1037                     # Close pipe fds.  Make sure we don't close the same

  1038                     # fd more than once, or standard fds.

  1039                     if p2cread and p2cread not in (0,):

  1040                         os.close(p2cread)

  1041                     if c2pwrite and c2pwrite not in (p2cread, 1):

  1042                         os.close(c2pwrite)

  1043                     if errwrite and errwrite not in (p2cread, c2pwrite, 2):

  1044                         os.close(errwrite)

  1045 

  1046                     # Close all other fds, if asked for

  1047                     if close_fds:

  1048                         self._close_fds(but=errpipe_write)

  1049 

  1050                     if cwd is not None:

  1051                         os.chdir(cwd)

  1052 

  1053                     if preexec_fn:

  1054                         apply(preexec_fn)

  1055 

  1056                     if env is None:

  1057                         os.execvp(executable, args)

  1058                     else:

  1059                         os.execvpe(executable, args, env)

  1060 

  1061                 except:

  1062                     exc_type, exc_value, tb = sys.exc_info()

  1063                     # Save the traceback and attach it to the exception object

  1064                     exc_lines = traceback.format_exception(exc_type,

  1065                                                            exc_value,

  1066                                                            tb)

  1067                     exc_value.child_traceback = ''.join(exc_lines)

  1068                     os.write(errpipe_write, pickle.dumps(exc_value))

  1069 

  1070                 # This exitcode won't be reported to applications, so it

  1071                 # really doesn't matter what we return.

  1072                 os._exit(255)

  1073 

  1074             # Parent

  1075             if gc_was_enabled:

  1076                 gc.enable()

  1077             os.close(errpipe_write)

  1078             if p2cread and p2cwrite:

  1079                 os.close(p2cread)

  1080             if c2pwrite and c2pread:

  1081                 os.close(c2pwrite)

  1082             if errwrite and errread:

  1083                 os.close(errwrite)

  1084 

  1085             # Wait for exec to fail or succeed; possibly raising exception

  1086             data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB

  1087             os.close(errpipe_read)

  1088             if data != "":

  1089                 os.waitpid(self.pid, 0)

  1090                 child_exception = pickle.loads(data)

  1091                 raise child_exception

  1092 

  1093 

  1094         def _handle_exitstatus(self, sts):

  1095             if os.WIFSIGNALED(sts):

  1096                 self.returncode = -os.WTERMSIG(sts)

  1097             elif os.WIFEXITED(sts):

  1098                 self.returncode = os.WEXITSTATUS(sts)

  1099             else:

  1100                 # Should never happen

  1101                 raise RuntimeError("Unknown child exit status!")

  1102 

  1103 

  1104         def poll(self, _deadstate=None):

  1105             """Check if child process has terminated.  Returns returncode

  1106             attribute."""

  1107             if self.returncode is None:

  1108                 try:

  1109                     pid, sts = os.waitpid(self.pid, os.WNOHANG)

  1110                     if pid == self.pid:

  1111                         self._handle_exitstatus(sts)

  1112                 except os.error:

  1113                     if _deadstate is not None:

  1114                         self.returncode = _deadstate

  1115             return self.returncode

  1116 

  1117 

  1118         def wait(self):

  1119             """Wait for child process to terminate.  Returns returncode

  1120             attribute."""

  1121             if self.returncode is None:

  1122                 pid, sts = os.waitpid(self.pid, 0)

  1123                 self._handle_exitstatus(sts)

  1124             return self.returncode

  1125 

  1126 

  1127         def _communicate(self, input):

  1128             read_set = []

  1129             write_set = []

  1130             stdout = None # Return

  1131             stderr = None # Return

  1132 

  1133             if self.stdin:

  1134                 # Flush stdio buffer.  This might block, if the user has

  1135                 # been writing to .stdin in an uncontrolled fashion.