Python subprocess Module

subprocess is a module in the Python standard library for creating and managing child processes.

subprocess allows you to execute external commands in Python programs and interact with those commands.

Through the subprocess module, you can execute system commands, call other programs, and obtain their output or error information.

Why Use the subprocess Module?

In Python, we sometimes need to execute system commands or call other programs to accomplish specific tasks. For example, you might need to run a shell command, launch an external application, or interact with a command-line tool. The subprocess module provides a safe and flexible way to handle these requirements.

Compared to the older os.system() or os.popen(), the subprocess module provides more powerful features and better control. It allows you to manage the input, output, and error streams of child processes more finely, and can handle more complex scenarios.

---

Core Features of the subprocess Module

1. Executing External Commands

subprocess.run() is one of the most commonly used functions in the subprocess module. It executes an external command and waits for the command to complete. Here is a simple example:

Example

import subprocess

# Execute a simple shell command
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)

# Print the command's output
print(result.stdout)

In this example, subprocess.run() executes the ls -l command and captures the output into result.stdout.

2. Handling Input and Output

The subprocess module allows you to control the input, output, and error streams of child processes. You can pass data to the child process’s standard input, or read data from the child process’s standard output and standard error. Here is an example:

Example

import subprocess

# Execute a command and pass input to the child process
result = subprocess.run(['grep', 'python'], input='hello\npython\nworld', capture_output=True, text=True)

# Print the command's output
print(result.stdout)

In this example, subprocess.run() executes the grep python command and passes the string 'hello\npython\nworld' as input to the child process.

3. Handling Errors

The subprocess module also allows you to handle errors from child processes. If a child process returns a non-zero exit status code, subprocess.run() will raise a CalledProcessError exception. You can check result.returncode to get the child process’s exit status code.

Example

import subprocess

try:
    result = subprocess.run(['ls', 'nonexistent_file'], capture_output=True, text=True, check=True)
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")
    print(f"Error output: {e.stderr}")

In this example, subprocess.run() executes the ls nonexistent_file command. Since the file does not exist, the command fails and raises a CalledProcessError exception.

---

Advanced Usage of the subprocess Module

1. Using the Popen Class

The subprocess.Popen class provides a lower-level interface, allowing you to control child processes more flexibly. You can use Popen to start a child process and run it in the background, or interact with it.

Example

import subprocess

# Start a child process
process = subprocess.Popen(['ping', 'google.com'], stdout=subprocess.PIPE, text=True)

# Read the child process's output
while True:
    output = process.stdout.readline()
    if output == '' and process.poll() is not None:
        break
    if output:
        print(output.strip())

# Get the child process's exit status code
return_code = process.poll()
print(f"Process finished with return code {return_code}")

In this example, subprocess.Popen starts a ping google.com command and runs it in the background. The program reads the child process’s output in a loop and gets its exit status code after the child process ends.

2. Using Pipes

The subprocess module allows you to use pipes to connect multiple commands together. You can use the output of one command as the input of another command.

Example

import subprocess

# Use pipes to connect two commands
p1 = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['grep', 'py'], stdin=p1.stdout, stdout=subprocess.PIPE, text=True)

# Get the final output
output = p2.communicate()[0]
print(output)

In this example, the output of the ls -l command is passed to the grep py command, and the final output contains files or directories containing py.

---

Common Methods, Classes, and Parameters of the subprocess Module

Below is a description of the common methods, classes, and parameters of the Python subprocess module, including feature descriptions and examples:

Core Methods of the subprocess Module

Method	Description	Example
subprocess.run()	Execute command and wait for completion (recommended)	subprocess.run(["ls", "-l"], capture_output=True, text=True)
subprocess.Popen()	Create a child process (low-level control)	proc = subprocess.Popen(["ping", "google.com"], stdout=subprocess.PIPE)
subprocess.call()	Execute command and return exit code (legacy)	exit_code = subprocess.call(["python", "--version"])
subprocess.check_call()	Execute command, raise exception on failure	subprocess.check_call(["git", "commit"])
subprocess.check_output()	Execute command and return output (legacy)	output = subprocess.check_output(["date"], text=True)

subprocess.CompletedProcess Object Attributes (return value of the run() method)

Attribute	Description
args	The command argument list executed
returncode	Process exit status code (0 indicates success)
stdout	Standard output content (if capture_output is set)
stderr	Standard error content (if capture_output is set)

Common Methods/Attributes of the subprocess.Popen Class

Method/Attribute	Description	Example
poll()	Check if process has terminated (returns None if running)	if proc.poll() is None: print("Running")
wait()	Block and wait for process to end	proc.wait()
communicate()	Interactive input/output	stdout, stderr = proc.communicate(input="data")
terminate()	Send termination signal (SIGTERM)	proc.terminate()
kill()	Force terminate process (SIGKILL)	proc.kill()
stdin	The process’s standard input stream	proc.stdin.write("input")
stdout	The process’s standard output stream	print(proc.stdout.read())
stderr	The process’s standard error stream	errors = proc.stderr.read()

Common Parameter Descriptions (applicable to run() and Popen())

Parameter	Description	Example Value
args	Command (list or string)	["ls", "-l"] or "ls -l"
stdin	Standard input configuration	subprocess.PIPE (pipe), None (inherit)
stdout	Standard output configuration	subprocess.PIPE, open('log.txt', 'w')
stderr	Standard error configuration	subprocess.STDOUT (merge to stdout)
shell	Whether to execute through Shell	True (supports string commands)
cwd	Working directory path	"/tmp"
env	Custom environment variables	{"PATH": "/usr/bin"}
timeout	Timeout duration (seconds)	30
text	Whether input/output is string (not bytes)	True

Examples

Execute command and capture output:

Example

result = subprocess.run(["echo", "Hello"], capture_output=True, text=True)
print(result.stdout)  # Output: "Hello\n"

Execute complex commands through Shell:

subprocess.run("grep 'error' log.txt | wc -l", shell=True, check=True)

Get real-time output stream:

Example

proc = subprocess.Popen(["tail", "-f", "log.txt"], stdout=subprocess.PIPE)
while True:
    line = proc.stdout.readline()
    if not line: break
    print(line.decode().strip())

Timeout control:

Example

try:
    subprocess.run(["sleep", "10"], timeout=5)
except subprocess.TimeoutExpired:
    print("Command timed out!")