Package cheesecake :: Module subprocess
[hide private]
[frames] | no frames]

Source Code for Module cheesecake.subprocess

   1  # subprocess - Subprocesses with accessible I/O streams 
   2  # 
   3  # For more information about this module, see PEP 324. 
   4  # 
   5  # Copyright (c) 2003-2004 by Peter Astrand <astrand@lysator.liu.se> 
   6  # 
   7  # By obtaining, using, and/or copying this software and/or its 
   8  # associated documentation, you agree that you have read, understood, 
   9  # and will comply with the following terms and conditions: 
  10  # 
  11  # Permission to use, copy, modify, and distribute this software and 
  12  # its associated documentation for any purpose and without fee is 
  13  # hereby granted, provided that the above copyright notice appears in 
  14  # all copies, and that both that copyright notice and this permission 
  15  # notice appear in supporting documentation, and that the name of the 
  16  # author not be used in advertising or publicity pertaining to 
  17  # distribution of the software without specific, written prior 
  18  # permission. 
  19  # 
  20  # THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, 
  21  # INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. 
  22  # IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR 
  23  # CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS 
  24  # OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, 
  25  # NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION 
  26  # WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 
  27   
  28  r"""subprocess - Subprocesses with accessible I/O streams 
  29   
  30  This module allows you to spawn processes, connect to their 
  31  input/output/error pipes, and obtain their return codes.  This module 
  32  intends to replace several other, older modules and functions, like: 
  33   
  34  os.system 
  35  os.spawn* 
  36  os.popen* 
  37  popen2.* 
  38  commands.* 
  39   
  40  Information about how the subprocess module can be used to replace these 
  41  modules and functions can be found below. 
  42   
  43   
  44   
  45  Using the subprocess module 
  46  =========================== 
  47  This module defines one class called Popen: 
  48   
  49  class Popen(args, bufsize=0, executable=None, 
  50              stdin=None, stdout=None, stderr=None, 
  51              preexec_fn=None, close_fds=False, shell=False, 
  52              cwd=None, env=None, universal_newlines=False, 
  53              startupinfo=None, creationflags=0): 
  54   
  55   
  56  Arguments are: 
  57   
  58  args should be a string, or a sequence of program arguments.  The 
  59  program to execute is normally the first item in the args sequence or 
  60  string, but can be explicitly set by using the executable argument. 
  61   
  62  On UNIX, with shell=False (default): In this case, the Popen class 
  63  uses os.execvp() to execute the child program.  args should normally 
  64  be a sequence.  A string will be treated as a sequence with the string 
  65  as the only item (the program to execute). 
  66   
  67  On UNIX, with shell=True: If args is a string, it specifies the 
  68  command string to execute through the shell.  If args is a sequence, 
  69  the first item specifies the command string, and any additional items 
  70  will be treated as additional shell arguments. 
  71   
  72  On Windows: the Popen class uses CreateProcess() to execute the child 
  73  program, which operates on strings.  If args is a sequence, it will be 
  74  converted to a string using the list2cmdline method.  Please note that 
  75  not all MS Windows applications interpret the command line the same 
  76  way: The list2cmdline is designed for applications using the same 
  77  rules as the MS C runtime. 
  78   
  79  bufsize, if given, has the same meaning as the corresponding argument 
  80  to the built-in open() function: 0 means unbuffered, 1 means line 
  81  buffered, any other positive value means use a buffer of 
  82  (approximately) that size.  A negative bufsize means to use the system 
  83  default, which usually means fully buffered.  The default value for 
  84  bufsize is 0 (unbuffered). 
  85   
  86  stdin, stdout and stderr specify the executed programs' standard 
  87  input, standard output and standard error file handles, respectively. 
  88  Valid values are PIPE, an existing file descriptor (a positive 
  89  integer), an existing file object, and None.  PIPE indicates that a 
  90  new pipe to the child should be created.  With None, no redirection 
  91  will occur; the child's file handles will be inherited from the 
  92  parent.  Additionally, stderr can be STDOUT, which indicates that the 
  93  stderr data from the applications should be captured into the same 
  94  file handle as for stdout. 
  95   
  96  If preexec_fn is set to a callable object, this object will be called 
  97  in the child process just before the child is executed. 
  98   
  99  If close_fds is true, all file descriptors except 0, 1 and 2 will be 
 100  closed before the child process is executed. 
 101   
 102  if shell is true, the specified command will be executed through the 
 103  shell. 
 104   
 105  If cwd is not None, the current directory will be changed to cwd 
 106  before the child is executed. 
 107   
 108  If env is not None, it defines the environment variables for the new 
 109  process. 
 110   
 111  If universal_newlines is true, the file objects stdout and stderr are 
 112  opened as a text files, but lines may be terminated by any of '\n', 
 113  the Unix end-of-line convention, '\r', the Macintosh convention or 
 114  '\r\n', the Windows convention.  All of these external representations 
 115  are seen as '\n' by the Python program.  Note: This feature is only 
 116  available if Python is built with universal newline support (the 
 117  default).  Also, the newlines attribute of the file objects stdout, 
 118  stdin and stderr are not updated by the communicate() method. 
 119   
 120  The startupinfo and creationflags, if given, will be passed to the 
 121  underlying CreateProcess() function.  They can specify things such as 
 122  appearance of the main window and priority for the new process. 
 123  (Windows only) 
 124   
 125   
 126  This module also defines two shortcut functions: 
 127   
 128  call(*args, **kwargs): 
 129      Run command with arguments.  Wait for command to complete, then 
 130      return the returncode attribute. 
 131   
 132      The arguments are the same as for the Popen constructor.  Example: 
 133   
 134      retcode = call(["ls", "-l"]) 
 135   
 136   
 137  Exceptions 
 138  ---------- 
 139  Exceptions raised in the child process, before the new program has 
 140  started to execute, will be re-raised in the parent.  Additionally, 
 141  the exception object will have one extra attribute called 
 142  'child_traceback', which is a string containing traceback information 
 143  from the childs point of view. 
 144   
 145  The most common exception raised is OSError.  This occurs, for 
 146  example, when trying to execute a non-existent file.  Applications 
 147  should prepare for OSErrors. 
 148   
 149  A ValueError will be raised if Popen is called with invalid arguments. 
 150   
 151   
 152  Security 
 153  -------- 
 154  Unlike some other popen functions, this implementation will never call 
 155  /bin/sh implicitly.  This means that all characters, including shell 
 156  metacharacters, can safely be passed to child processes. 
 157   
 158   
 159  Popen objects 
 160  ============= 
 161  Instances of the Popen class have the following methods: 
 162   
 163  poll() 
 164      Check if child process has terminated.  Returns returncode 
 165      attribute. 
 166   
 167  wait() 
 168      Wait for child process to terminate.  Returns returncode attribute. 
 169   
 170  communicate(input=None) 
 171      Interact with process: Send data to stdin.  Read data from stdout 
 172      and stderr, until end-of-file is reached.  Wait for process to 
 173      terminate.  The optional stdin argument should be a string to be 
 174      sent to the child process, or None, if no data should be sent to 
 175      the child. 
 176   
 177      communicate() returns a tuple (stdout, stderr). 
 178   
 179      Note: The data read is buffered in memory, so do not use this 
 180      method if the data size is large or unlimited. 
 181   
 182  The following attributes are also available: 
 183   
 184  stdin 
 185      If the stdin argument is PIPE, this attribute is a file object 
 186      that provides input to the child process.  Otherwise, it is None. 
 187   
 188  stdout 
 189      If the stdout argument is PIPE, this attribute is a file object 
 190      that provides output from the child process.  Otherwise, it is 
 191      None. 
 192   
 193  stderr 
 194      If the stderr argument is PIPE, this attribute is file object that 
 195      provides error output from the child process.  Otherwise, it is 
 196      None. 
 197   
 198  pid 
 199      The process ID of the child process. 
 200   
 201  returncode 
 202      The child return code.  A None value indicates that the process 
 203      hasn't terminated yet.  A negative value -N indicates that the 
 204      child was terminated by signal N (UNIX only). 
 205   
 206   
 207  Replacing older functions with the subprocess module 
 208  ==================================================== 
 209  In this section, "a ==> b" means that b can be used as a replacement 
 210  for a. 
 211   
 212  Note: All functions in this section fail (more or less) silently if 
 213  the executed program cannot be found; this module raises an OSError 
 214  exception. 
 215   
 216  In the following examples, we assume that the subprocess module is 
 217  imported with "from subprocess import *". 
 218   
 219   
 220  Replacing /bin/sh shell backquote 
 221  --------------------------------- 
 222  output=`mycmd myarg` 
 223  ==> 
 224  output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] 
 225   
 226   
 227  Replacing shell pipe line 
 228  ------------------------- 
 229  output=`dmesg | grep hda` 
 230  ==> 
 231  p1 = Popen(["dmesg"], stdout=PIPE) 
 232  p2 = Popen(["grep", "hda"], stdin=p1.stdout) 
 233  output = p2.communicate()[0] 
 234   
 235   
 236  Replacing os.system() 
 237  --------------------- 
 238  sts = os.system("mycmd" + " myarg") 
 239  ==> 
 240  p = Popen("mycmd" + " myarg", shell=True) 
 241  sts = os.waitpid(p.pid, 0) 
 242   
 243  Note: 
 244   
 245  * Calling the program through the shell is usually not required. 
 246   
 247  * It's easier to look at the returncode attribute than the 
 248    exitstatus. 
 249   
 250  A more real-world example would look like this: 
 251   
 252  try: 
 253      retcode = call("mycmd" + " myarg", shell=True) 
 254      if retcode < 0: 
 255          print >>sys.stderr, "Child was terminated by signal", -retcode 
 256      else: 
 257          print >>sys.stderr, "Child returned", retcode 
 258  except OSError, e: 
 259      print >>sys.stderr, "Execution failed:", e 
 260   
 261   
 262  Replacing os.spawn* 
 263  ------------------- 
 264  P_NOWAIT example: 
 265   
 266  pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") 
 267  ==> 
 268  pid = Popen(["/bin/mycmd", "myarg"]).pid 
 269   
 270   
 271  P_WAIT example: 
 272   
 273  retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") 
 274  ==> 
 275  retcode = call(["/bin/mycmd", "myarg"]) 
 276   
 277   
 278  Vector example: 
 279   
 280  os.spawnvp(os.P_NOWAIT, path, args) 
 281  ==> 
 282  Popen([path] + args[1:]) 
 283   
 284   
 285  Environment example: 
 286   
 287  os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) 
 288  ==> 
 289  Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) 
 290   
 291   
 292  Replacing os.popen* 
 293  ------------------- 
 294  pipe = os.popen(cmd, mode='r', bufsize) 
 295  ==> 
 296  pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout 
 297   
 298  pipe = os.popen(cmd, mode='w', bufsize) 
 299  ==> 
 300  pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin 
 301   
 302   
 303  (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) 
 304  ==> 
 305  p = Popen(cmd, shell=True, bufsize=bufsize, 
 306            stdin=PIPE, stdout=PIPE, close_fds=True) 
 307  (child_stdin, child_stdout) = (p.stdin, p.stdout) 
 308   
 309   
 310  (child_stdin, 
 311   child_stdout, 
 312   child_stderr) = os.popen3(cmd, mode, bufsize) 
 313  ==> 
 314  p = Popen(cmd, shell=True, bufsize=bufsize, 
 315            stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) 
 316  (child_stdin, 
 317   child_stdout, 
 318   child_stderr) = (p.stdin, p.stdout, p.stderr) 
 319   
 320   
 321  (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) 
 322  ==> 
 323  p = Popen(cmd, shell=True, bufsize=bufsize, 
 324            stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) 
 325  (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) 
 326   
 327   
 328  Replacing popen2.* 
 329  ------------------ 
 330  Note: If the cmd argument to popen2 functions is a string, the command 
 331  is executed through /bin/sh.  If it is a list, the command is directly 
 332  executed. 
 333   
 334  (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) 
 335  ==> 
 336  p = Popen(["somestring"], shell=True, bufsize=bufsize 
 337            stdin=PIPE, stdout=PIPE, close_fds=True) 
 338  (child_stdout, child_stdin) = (p.stdout, p.stdin) 
 339   
 340   
 341  (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) 
 342  ==> 
 343  p = Popen(["mycmd", "myarg"], bufsize=bufsize, 
 344            stdin=PIPE, stdout=PIPE, close_fds=True) 
 345  (child_stdout, child_stdin) = (p.stdout, p.stdin) 
 346   
 347  The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, 
 348  except that: 
 349   
 350  * subprocess.Popen raises an exception if the execution fails 
 351  * the capturestderr argument is replaced with the stderr argument. 
 352  * stdin=PIPE and stdout=PIPE must be specified. 
 353  * popen2 closes all filedescriptors by default, but you have to specify 
 354    close_fds=True with subprocess.Popen. 
 355   
 356   
 357  """ 
 358   
 359  import sys 
 360  mswindows = (sys.platform == "win32") 
 361   
 362  import os 
 363  import types 
 364  import traceback 
 365   
 366  if mswindows: 
 367      import threading 
 368      import msvcrt 
 369      if 0: # <-- change this to use pywin32 instead of the _subprocess driver 
 370          import pywintypes 
 371          from win32api import GetStdHandle, STD_INPUT_HANDLE, \ 
 372                               STD_OUTPUT_HANDLE, STD_ERROR_HANDLE 
 373          from win32api import GetCurrentProcess, DuplicateHandle, \ 
 374                               GetModuleFileName, GetVersion 
 375          from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE 
 376          from win32pipe import CreatePipe 
 377          from win32process import CreateProcess, STARTUPINFO, \ 
 378                                   GetExitCodeProcess, STARTF_USESTDHANDLES, \ 
 379                                   STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE 
 380          from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0 
 381      else: 
 382          from _subprocess import * 
 383          class STARTUPINFO: 
 384              dwFlags = 0 
 385              hStdInput = None 
 386              hStdOutput = None 
 387              hStdError = None 
388 class pywintypes: 389 error = IOError 390 else: 391 import select 392 import errno 393 import fcntl 394 import pickle 395 396 __all__ = ["Popen", "PIPE", "STDOUT", "call", "ProcessError"] 397 398 try: 399 MAXFD = os.sysconf("SC_OPEN_MAX") 400 except: 401 MAXFD = 256 402 403 # True/False does not exist on 2.2.0 404 try: 405 False 406 except NameError: 407 False = 0 408 True = 1 409 410 _active = [] 411
412 -def _cleanup():
413 for inst in _active[:]: 414 inst.poll()
415 416 PIPE = -1 417 STDOUT = -2 418 419
420 -def call(*args, **kwargs):
421 """Run command with arguments. Wait for command to complete, then 422 return the returncode attribute. 423 424 The arguments are the same as for the Popen constructor. Example: 425 426 retcode = call(["ls", "-l"]) 427 """ 428 return Popen(*args, **kwargs).wait()
429 430
431 -def list2cmdline(seq):
432 """ 433 Translate a sequence of arguments into a command line 434 string, using the same rules as the MS C runtime: 435 436 1) Arguments are delimited by white space, which is either a 437 space or a tab. 438 439 2) A string surrounded by double quotation marks is 440 interpreted as a single argument, regardless of white space 441 contained within. A quoted string can be embedded in an 442 argument. 443 444 3) A double quotation mark preceded by a backslash is 445 interpreted as a literal double quotation mark. 446 447 4) Backslashes are interpreted literally, unless they 448 immediately precede a double quotation mark. 449 450 5) If backslashes immediately precede a double quotation mark, 451 every pair of backslashes is interpreted as a literal 452 backslash. If the number of backslashes is odd, the last 453 backslash escapes the next double quotation mark as 454 described in rule 3. 455 """ 456 457 # See 458 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp 459 result = [] 460 needquote = False 461 for arg in seq: 462 bs_buf = [] 463 464 # Add a space to separate this argument from the others 465 if result: 466 result.append(' ') 467 468 needquote = (" " in arg) or ("\t" in arg) 469 if needquote: 470 result.append('"') 471 472 for c in arg: 473 if c == '\\': 474 # Don't know if we need to double yet. 475 bs_buf.append(c) 476 elif c == '"': 477 # Double backspaces. 478 result.append('\\' * len(bs_buf)*2) 479 bs_buf = [] 480 result.append('\\"') 481 else: 482 # Normal char 483 if bs_buf: 484 result.extend(bs_buf) 485 bs_buf = [] 486 result.append(c) 487 488 # Add remaining backspaces, if any. 489 if bs_buf: 490 result.extend(bs_buf) 491 492 if needquote: 493 result.append('"') 494 495 return ''.join(result)
496
497 -class ProcessError(Exception):
498 """This exception is raised when there is an error calling 499 a subprocess.""" 500 pass
501
502 -class Popen(object):
503 - def __init__(self, args, bufsize=0, executable=None, 504 stdin=None, stdout=None, stderr=None, 505 preexec_fn=None, close_fds=False, shell=False, 506 cwd=None, env=None, universal_newlines=False, 507 startupinfo=None, creationflags=0):
508 """Create new Popen instance.""" 509 _cleanup() 510 511 if mswindows: 512 if preexec_fn is not None: 513 raise ValueError("preexec_fn is not supported on Windows " 514 "platforms") 515 if close_fds: 516 raise ValueError("close_fds is not supported on Windows " 517 "platforms") 518 else: 519 # POSIX 520 if startupinfo is not None: 521 raise ValueError("startupinfo is only supported on Windows " 522 "platforms") 523 if creationflags != 0: 524 raise ValueError("creationflags is only supported on Windows " 525 "platforms") 526 527 self.stdin = None 528 self.stdout = None 529 self.stderr = None 530 self.pid = None 531 self.returncode = None 532 self.universal_newlines = universal_newlines 533 534 # Input and output objects. The general principle is like 535 # this: 536 # 537 # Parent Child 538 # ------ ----- 539 # p2cwrite ---stdin---> p2cread 540 # c2pread <--stdout--- c2pwrite 541 # errread <--stderr--- errwrite 542 # 543 # On POSIX, the child objects are file descriptors. On 544 # Windows, these are Windows file handles. The parent objects 545 # are file descriptors on both platforms. The parent objects 546 # are None when not using PIPEs. The child objects are None 547 # when not redirecting. 548 549 (p2cread, p2cwrite, 550 c2pread, c2pwrite, 551 errread, errwrite) = self._get_handles(stdin, stdout, stderr) 552 553 self._execute_child(args, executable, preexec_fn, close_fds, 554 cwd, env, universal_newlines, 555 startupinfo, creationflags, shell, 556 p2cread, p2cwrite, 557 c2pread, c2pwrite, 558 errread, errwrite) 559 560 if p2cwrite: 561 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize) 562 if c2pread: 563 if universal_newlines: 564 self.stdout = os.fdopen(c2pread, 'rU', bufsize) 565 else: 566 self.stdout = os.fdopen(c2pread, 'rb', bufsize) 567 if errread: 568 if universal_newlines: 569 self.stderr = os.fdopen(errread, 'rU', bufsize) 570 else: 571 self.stderr = os.fdopen(errread, 'rb', bufsize) 572 573 _active.append(self)
574 575
576 - def _translate_newlines(self, data):
577 data = data.replace("\r\n", "\n") 578 data = data.replace("\r", "\n") 579 return data
580 581 582 if mswindows: 583 # 584 # Windows methods 585 # 586 def _get_handles(self, stdin, stdout, stderr): 587 """Construct and return tupel with IO objects: 588 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 589 """ 590 if stdin == None and stdout == None and stderr == None: 591 return (None, None, None, None, None, None) 592 593 p2cread, p2cwrite = None, None 594 c2pread, c2pwrite = None, None 595 errread, errwrite = None, None 596 597 if stdin == None: 598 p2cread = GetStdHandle(STD_INPUT_HANDLE) 599 elif stdin == PIPE: 600 p2cread, p2cwrite = CreatePipe(None, 0) 601 # Detach and turn into fd 602 p2cwrite = p2cwrite.Detach() 603 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0) 604 elif type(stdin) == types.IntType: 605 p2cread = msvcrt.get_osfhandle(stdin) 606 else: 607 # Assuming file-like object 608 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 609 p2cread = self._make_inheritable(p2cread) 610 611 if stdout == None: 612 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE) 613 elif stdout == PIPE: 614 c2pread, c2pwrite = CreatePipe(None, 0) 615 # Detach and turn into fd 616 c2pread = c2pread.Detach() 617 c2pread = msvcrt.open_osfhandle(c2pread, 0) 618 elif type(stdout) == types.IntType: 619 c2pwrite = msvcrt.get_osfhandle(stdout) 620 else: 621 # Assuming file-like object 622 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 623 c2pwrite = self._make_inheritable(c2pwrite) 624 625 if stderr == None: 626 errwrite = GetStdHandle(STD_ERROR_HANDLE) 627 elif stderr == PIPE: 628 errread, errwrite = CreatePipe(None, 0) 629 # Detach and turn into fd 630 errread = errread.Detach() 631 errread = msvcrt.open_osfhandle(errread, 0) 632 elif stderr == STDOUT: 633 errwrite = c2pwrite 634 elif type(stderr) == types.IntType: 635 errwrite = msvcrt.get_osfhandle(stderr) 636 else: 637 # Assuming file-like object 638 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 639 errwrite = self._make_inheritable(errwrite) 640 641 return (p2cread, p2cwrite, 642 c2pread, c2pwrite, 643 errread, errwrite)
644 645 646 def _make_inheritable(self, handle): 647 """Return a duplicate of handle, which is inheritable""" 648 return DuplicateHandle(GetCurrentProcess(), handle, 649 GetCurrentProcess(), 0, 1, 650 DUPLICATE_SAME_ACCESS) 651 652 653 def _find_w9xpopen(self): 654 """Find and return absolut path to w9xpopen.exe""" 655 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)), 656 "w9xpopen.exe") 657 if not os.path.exists(w9xpopen): 658 # Eeek - file-not-found - possibly an embedding 659 # situation - see if we can locate it in sys.exec_prefix 660 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix), 661 "w9xpopen.exe") 662 if not os.path.exists(w9xpopen): 663 raise RuntimeError("Cannot locate w9xpopen.exe, which is " 664 "needed for Popen to work with your " 665 "shell or platform.") 666 return w9xpopen 667 668 669 def _execute_child(self, args, executable, preexec_fn, close_fds, 670 cwd, env, universal_newlines, 671 startupinfo, creationflags, shell, 672 p2cread, p2cwrite, 673 c2pread, c2pwrite, 674 errread, errwrite): 675 """Execute program (MS Windows version)""" 676 677 if not isinstance(args, types.StringTypes): 678 args = list2cmdline(args) 679 680 # Process startup details 681 default_startupinfo = STARTUPINFO() 682 if startupinfo == None: 683 startupinfo = default_startupinfo 684 if not None in (p2cread, c2pwrite, errwrite): 685 startupinfo.dwFlags |= STARTF_USESTDHANDLES 686 startupinfo.hStdInput = p2cread 687 startupinfo.hStdOutput = c2pwrite 688 startupinfo.hStdError = errwrite 689 690 if shell: 691 default_startupinfo.dwFlags |= STARTF_USESHOWWINDOW 692 default_startupinfo.wShowWindow = SW_HIDE 693 comspec = os.environ.get("COMSPEC", "cmd.exe") 694 args = comspec + " /c " + args 695 if (GetVersion() >= 0x80000000L or 696 os.path.basename(comspec).lower() == "command.com"): 697 # Win9x, or using command.com on NT. We need to 698 # use the w9xpopen intermediate program. For more 699 # information, see KB Q150956 700 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp) 701 w9xpopen = self._find_w9xpopen() 702 args = '"%s" %s' % (w9xpopen, args) 703 # Not passing CREATE_NEW_CONSOLE has been known to 704 # cause random failures on win9x. Specifically a 705 # dialog: "Your program accessed mem currently in 706 # use at xxx" and a hopeful warning about the 707 # stability of your system. Cost is Ctrl+C wont 708 # kill children. 709 creationflags |= CREATE_NEW_CONSOLE 710 711 # Start the process 712 try: 713 hp, ht, pid, tid = CreateProcess(executable, args, 714 # no special security 715 None, None, 716 # must inherit handles to pass std 717 # handles 718 1, 719 creationflags, 720 env, 721 cwd, 722 startupinfo) 723 except pywintypes.error, e: 724 # Translate pywintypes.error to WindowsError, which is 725 # a subclass of OSError. FIXME: We should really 726 # translate errno using _sys_errlist (or simliar), but 727 # how can this be done from Python? 728 raise WindowsError(*e.args) 729 730 # Retain the process handle, but close the thread handle 731 self._handle = hp 732 self.pid = pid 733 ht.Close() 734 735 # Child is launched. Close the parent's copy of those pipe 736 # handles that only the child should have open. You need 737 # to make sure that no handles to the write end of the 738 # output pipe are maintained in this process or else the 739 # pipe will not close when the child process exits and the 740 # ReadFile will hang. 741 if p2cread != None: 742 p2cread.Close() 743 if c2pwrite != None: 744 c2pwrite.Close() 745 if errwrite != None: 746 errwrite.Close() 747 748 749 def poll(self): 750 """Check if child process has terminated. Returns returncode 751 attribute.""" 752 if self.returncode == None: 753 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0: 754 self.returncode = GetExitCodeProcess(self._handle) 755 _active.remove(self) 756 return self.returncode 757 758 759 def wait(self): 760 """Wait for child process to terminate. Returns returncode 761 attribute.""" 762 if self.returncode == None: 763 obj = WaitForSingleObject(self._handle, INFINITE) 764 self.returncode = GetExitCodeProcess(self._handle) 765 _active.remove(self) 766 return self.returncode 767 768 769 def _readerthread(self, fh, buffer): 770 buffer.append(fh.read()) 771 772 773 def communicate(self, input=None): 774 """Interact with process: Send data to stdin. Read data from 775 stdout and stderr, until end-of-file is reached. Wait for 776 process to terminate. The optional input argument should be a 777 string to be sent to the child process, or None, if no data 778 should be sent to the child. 779 780 communicate() returns a tuple (stdout, stderr).""" 781 stdout = None # Return 782 stderr = None # Return 783 784 if self.stdout: 785 stdout = [] 786 stdout_thread = threading.Thread(target=self._readerthread, 787 args=(self.stdout, stdout)) 788 stdout_thread.setDaemon(True) 789 stdout_thread.start() 790 if self.stderr: 791 stderr = [] 792 stderr_thread = threading.Thread(target=self._readerthread, 793 args=(self.stderr, stderr)) 794 stderr_thread.setDaemon(True) 795 stderr_thread.start() 796 797 if self.stdin: 798 if input != None: 799 self.stdin.write(input) 800 self.stdin.close() 801 802 if self.stdout: 803 stdout_thread.join() 804 if self.stderr: 805 stderr_thread.join() 806 807 # All data exchanged. Translate lists into strings. 808 if stdout != None: 809 stdout = stdout[0] 810 if stderr != None: 811 stderr = stderr[0] 812 813 # Translate newlines, if requested. We cannot let the file 814 # object do the translation: It is based on stdio, which is 815 # impossible to combine with select (unless forcing no 816 # buffering). 817 if self.universal_newlines and hasattr(open, 'newlines'): 818 if stdout: 819 stdout = self._translate_newlines(stdout) 820 if stderr: 821 stderr = self._translate_newlines(stderr) 822 823 self.wait() 824 return (stdout, stderr) 825 826 else: 827 # 828 # POSIX methods 829 # 830 def _get_handles(self, stdin, stdout, stderr): 831 """Construct and return tupel with IO objects: 832 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 833 """ 834 p2cread, p2cwrite = None, None 835 c2pread, c2pwrite = None, None 836 errread, errwrite = None, None 837 838 if stdin == None: 839 pass 840 elif stdin == PIPE: 841 p2cread, p2cwrite = os.pipe() 842 elif type(stdin) == types.IntType: 843 p2cread = stdin 844 else: 845 # Assuming file-like object 846 p2cread = stdin.fileno() 847 848 if stdout == None: 849 pass 850 elif stdout == PIPE: 851 c2pread, c2pwrite = os.pipe() 852 elif type(stdout) == types.IntType: 853 c2pwrite = stdout 854 else: 855 # Assuming file-like object 856 c2pwrite = stdout.fileno() 857 858 if stderr == None: 859 pass 860 elif stderr == PIPE: 861 errread, errwrite = os.pipe() 862 elif stderr == STDOUT: 863 errwrite = c2pwrite 864 elif type(stderr) == types.IntType: 865 errwrite = stderr 866 else: 867 # Assuming file-like object 868 errwrite = stderr.fileno() 869 870 return (p2cread, p2cwrite, 871 c2pread, c2pwrite, 872 errread, errwrite) 873 874 875 def _set_cloexec_flag(self, fd): 876 try: 877 cloexec_flag = fcntl.FD_CLOEXEC 878 except AttributeError: 879 cloexec_flag = 1 880 881 old = fcntl.fcntl(fd, fcntl.F_GETFD) 882 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag) 883 884 885 def _close_fds(self, but): 886 for i in range(3, MAXFD): 887 if i == but: 888 continue 889 try: 890 os.close(i) 891 except: 892 pass 893 894 895 def _execute_child(self, args, executable, preexec_fn, close_fds, 896 cwd, env, universal_newlines, 897 startupinfo, creationflags, shell, 898 p2cread, p2cwrite, 899 c2pread, c2pwrite, 900 errread, errwrite): 901 """Execute program (POSIX version)""" 902 903 if isinstance(args, types.StringTypes): 904 args = [args] 905 906 if shell: 907 args = ["/bin/sh", "-c"] + args 908 909 if executable == None: 910 executable = args[0] 911 912 # For transferring possible exec failure from child to parent 913 # The first char specifies the exception type: 0 means 914 # OSError, 1 means some other error. 915 errpipe_read, errpipe_write = os.pipe() 916 self._set_cloexec_flag(errpipe_write) 917 918 self.pid = os.fork() 919 if self.pid == 0: 920 # Child 921 try: 922 # Close parent's pipe ends 923 if p2cwrite: 924 os.close(p2cwrite) 925 if c2pread: 926 os.close(c2pread) 927 if errread: 928 os.close(errread) 929 os.close(errpipe_read) 930 931 # Dup fds for child 932 if p2cread: 933 os.dup2(p2cread, 0) 934 if c2pwrite: 935 os.dup2(c2pwrite, 1) 936 if errwrite: 937 os.dup2(errwrite, 2) 938 939 # Close pipe fds. Make sure we doesn't close the same 940 # fd more than once. 941 if p2cread: 942 os.close(p2cread) 943 if c2pwrite and c2pwrite not in (p2cread,): 944 os.close(c2pwrite) 945 if errwrite and errwrite not in (p2cread, c2pwrite): 946 os.close(errwrite) 947 948 # Close all other fds, if asked for 949 if close_fds: 950 self._close_fds(but=errpipe_write) 951 952 if cwd != None: 953 os.chdir(cwd) 954 955 if preexec_fn: 956 apply(preexec_fn) 957 958 if env == None: 959 os.execvp(executable, args) 960 else: 961 os.execvpe(executable, args, env) 962 963 except: 964 exc_type, exc_value, tb = sys.exc_info() 965 # Save the traceback and attach it to the exception object 966 exc_lines = traceback.format_exception(exc_type, 967 exc_value, 968 tb) 969 exc_value.child_traceback = ''.join(exc_lines) 970 os.write(errpipe_write, pickle.dumps(exc_value)) 971 972 # This exitcode won't be reported to applications, so it 973 # really doesn't matter what we return. 974 os._exit(255) 975 976 # Parent 977 os.close(errpipe_write) 978 if p2cread and p2cwrite: 979 os.close(p2cread) 980 if c2pwrite and c2pread: 981 os.close(c2pwrite) 982 if errwrite and errread: 983 os.close(errwrite) 984 985 # Wait for exec to fail or succeed; possibly raising exception 986 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB 987 os.close(errpipe_read) 988 if data != "": 989 child_exception = pickle.loads(data) 990 raise ProcessError, child_exception 991 992 993 def _handle_exitstatus(self, sts): 994 if os.WIFSIGNALED(sts): 995 self.returncode = -os.WTERMSIG(sts) 996 elif os.WIFEXITED(sts): 997 self.returncode = os.WEXITSTATUS(sts) 998 else: 999 # Should never happen 1000 raise RuntimeError("Unknown child exit status!") 1001 1002 _active.remove(self) 1003 1004 1005 def poll(self): 1006 """Check if child process has terminated. Returns returncode 1007 attribute.""" 1008 if self.returncode == None: 1009 try: 1010 pid, sts = os.waitpid(self.pid, os.WNOHANG) 1011 if pid == self.pid: 1012 self._handle_exitstatus(sts) 1013 except os.error: 1014 pass 1015 return self.returncode 1016 1017 1018 def wait(self): 1019 """Wait for child process to terminate. Returns returncode 1020 attribute.""" 1021 if self.returncode == None: 1022 pid, sts = os.waitpid(self.pid, 0) 1023 self._handle_exitstatus(sts) 1024 return self.returncode 1025 1026 1027 def communicate(self, input=None): 1028 """Interact with process: Send data to stdin. Read data from 1029 stdout and stderr, until end-of-file is reached. Wait for 1030 process to terminate. The optional input argument should be a 1031 string to be sent to the child process, or None, if no data 1032 should be sent to the child. 1033 1034 communicate() returns a tuple (stdout, stderr).""" 1035 read_set = [] 1036 write_set = [] 1037 stdout = None # Return 1038 stderr = None # Return 1039 1040 if self.stdin: 1041 # Flush stdio buffer. This might block, if the user has 1042 # been writing to .stdin in an uncontrolled fashion. 1043 self.stdin.flush() 1044 if input: 1045 write_set.append(self.stdin) 1046 else: 1047 self.stdin.close() 1048 if self.stdout: 1049 read_set.append(self.stdout) 1050 stdout = [] 1051 if self.stderr: 1052 read_set.append(self.stderr) 1053 stderr = [] 1054 1055 while read_set or write_set: 1056 rlist, wlist, xlist = select.select(read_set, write_set, []) 1057 1058 if self.stdin in wlist: 1059 # When select has indicated that the file is writable, 1060 # we can write up to PIPE_BUF bytes without risk 1061 # blocking. POSIX defines PIPE_BUF >= 512 1062 bytes_written = os.write(self.stdin.fileno(), input[:512]) 1063 input = input[bytes_written:] 1064 if not input: 1065 self.stdin.close() 1066 write_set.remove(self.stdin) 1067 1068 if self.stdout in rlist: 1069 data = os.read(self.stdout.fileno(), 1024) 1070 if data == "": 1071 self.stdout.close() 1072 read_set.remove(self.stdout) 1073 stdout.append(data) 1074 1075 if self.stderr in rlist: 1076 data = os.read(self.stderr.fileno(), 1024) 1077 if data == "": 1078 self.stderr.close() 1079 read_set.remove(self.stderr) 1080 stderr.append(data) 1081 1082 # All data exchanged. Translate lists into strings. 1083 if stdout != None: 1084 stdout = ''.join(stdout) 1085 if stderr != None: 1086 stderr = ''.join(stderr) 1087 1088 # Translate newlines, if requested. We cannot let the file 1089 # object do the translation: It is based on stdio, which is 1090 # impossible to combine with select (unless forcing no 1091 # buffering). 1092 if self.universal_newlines and hasattr(open, 'newlines'): 1093 if stdout: 1094 stdout = self._translate_newlines(stdout) 1095 if stderr: 1096 stderr = self._translate_newlines(stderr) 1097 1098 self.wait() 1099 return (stdout, stderr) 1100 1101
1102 -def _demo_posix():
1103 # 1104 # Example 1: Simple redirection: Get process list 1105 # 1106 plist = Popen(["ps"], stdout=PIPE).communicate()[0] 1107 print "Process list:" 1108 print plist 1109 1110 # 1111 # Example 2: Change uid before executing child 1112 # 1113 if os.getuid() == 0: 1114 p = Popen(["id"], preexec_fn=lambda: os.setuid(100)) 1115 p.wait() 1116 1117 # 1118 # Example 3: Connecting several subprocesses 1119 # 1120 print "Looking for 'hda'..." 1121 p1 = Popen(["dmesg"], stdout=PIPE) 1122 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 1123 print repr(p2.communicate()[0]) 1124 1125 # 1126 # Example 4: Catch execution error 1127 # 1128 print 1129 print "Trying a weird file..." 1130 try: 1131 print Popen(["/this/path/does/not/exist"]).communicate() 1132 except OSError, e: 1133 if e.errno == errno.ENOENT: 1134 print "The file didn't exist. I thought so..." 1135 print "Child traceback:" 1136 print e.child_traceback 1137 else: 1138 print "Error", e.errno 1139 else: 1140 print >>sys.stderr, "Gosh. No error."
1141 1142
1143 -def _demo_windows():
1144 # 1145 # Example 1: Connecting several subprocesses 1146 # 1147 print "Looking for 'PROMPT' in set output..." 1148 p1 = Popen("set", stdout=PIPE, shell=True) 1149 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE) 1150 print repr(p2.communicate()[0]) 1151 1152 # 1153 # Example 2: Simple execution of program 1154 # 1155 print "Executing calc..." 1156 p = Popen("calc") 1157 p.wait()
1158 1159 1160 if __name__ == "__main__": 1161 if mswindows: 1162 _demo_windows() 1163 else: 1164 _demo_posix() 1165