Go’s os/exec package gives you precise control over subprocess execution: capturing vs streaming output, timeouts, signal forwarding, environment control, and working directory. The patterns here are the foundation for build tools, deployment scripts, and any CLI that orchestrates external programs.
For shell-specific patterns (piping, script execution, injection prevention) see Go shell integration and scripting.
The Core API
exec.Command creates a *exec.Cmd representing a subprocess. It doesn’t start anything — you configure it first, then call one of three methods:
cmd.Run()— start and wait for completion, no output capturecmd.Output()— start, capture stdout, wait; returns[]bytecmd.CombinedOutput()— start, capture stdout+stderr combined, wait
For anything requiring a timeout, use exec.CommandContext and cancel the context:
// Always set a timeout for external commands — they can hang indefinitely
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "git", "clone", "https://github.com/example/repo")
out, err := cmd.Output()
if err != nil {
var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
return fmt.Errorf("git clone exited %d: %s", exitErr.ExitCode(), exitErr.Stderr)
}
if ctx.Err() != nil {
return fmt.Errorf("git clone timed out")
}
return fmt.Errorf("git clone: %w", err)
}
exec.ExitError wraps a non-zero exit code. It has an ExitCode() method and, for commands run with Output(), a Stderr field containing whatever the command wrote to stderr. Always check both.
Capturing vs Streaming Output
cmd.Output() buffers the entire output in memory — right for small outputs, wrong for long-running processes:
// ✅ Small output — capture it all
out, err := exec.CommandContext(ctx, "go", "version").Output()
// out is []byte{"go version go1.22.0 linux/amd64\n"}
For long-running processes (builds, tests, deployments), connect cmd.Stdout and cmd.Stderr directly to writers so output streams in real time:
func runWithLiveOutput(ctx context.Context, name string, args ...string) error {
cmd := exec.CommandContext(ctx, name, args...)
cmd.Stdout = os.Stdout // or any io.Writer: a log file, a buffer, etc.
cmd.Stderr = os.Stderr
return cmd.Run()
}
// To stream AND capture simultaneously, use io.MultiWriter
func runAndCapture(ctx context.Context, name string, args ...string) (string, error) {
var buf bytes.Buffer
cmd := exec.CommandContext(ctx, name, args...)
cmd.Stdout = io.MultiWriter(os.Stdout, &buf)
cmd.Stderr = os.Stderr
err := cmd.Run()
return buf.String(), err
}
Environment Control
By default, the subprocess inherits the parent’s environment. Override it for reproducibility:
cmd := exec.CommandContext(ctx, "go", "build", "-o", "myapp", ".")
// Start from parent environment, add/override specific variables
cmd.Env = append(os.Environ(),
"CGO_ENABLED=0",
"GOOS=linux",
"GOARCH=amd64",
fmt.Sprintf("GOPATH=%s", customGoPath),
)
// Or start with a clean environment (no parent vars)
cmd.Env = []string{
"PATH=/usr/local/go/bin:/usr/bin:/bin",
"HOME=/tmp",
"GOOS=linux",
}
cmd.Dir = "./myproject" // working directory for the subprocess
Signal Forwarding
When your Go process receives SIGTERM or SIGINT, child processes don’t automatically receive it. If you have a subprocess running, you need to forward the signal:
func runWithSignalForwarding(name string, args ...string) error {
cmd := exec.Command(name, args...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Stdin = os.Stdin
if err := cmd.Start(); err != nil {
return fmt.Errorf("start: %w", err)
}
// Channel to receive OS signals
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
done := make(chan error, 1)
go func() { done <- cmd.Wait() }()
select {
case sig := <-sigCh:
// Forward to the child process
cmd.Process.Signal(sig)
signal.Stop(sigCh)
return <-done // wait for child to exit
case err := <-done:
signal.Stop(sigCh)
return err
}
}
On Unix, you can also create a process group and signal the entire group — this ensures all child processes of the subprocess also receive the signal:
cmd := exec.Command(name, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} // new process group
// To kill the group:
syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM) // negative PID = process group
Retry Logic
External commands fail transiently — network issues, resource contention, transient errors. A retry wrapper with exponential backoff:
func runWithRetry(ctx context.Context, maxAttempts int, name string, args ...string) error {
var lastErr error
delay := 500 * time.Millisecond
for attempt := 1; attempt <= maxAttempts; attempt++ {
if ctx.Err() != nil {
return ctx.Err()
}
cmd := exec.CommandContext(ctx, name, args...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err == nil {
return nil
} else {
lastErr = err
// Don't retry context cancellation
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return err
}
if attempt < maxAttempts {
slog.Warn("command failed, retrying",
slog.String("cmd", name),
slog.Int("attempt", attempt),
slog.Duration("delay", delay),
slog.Any("error", err),
)
select {
case <-time.After(delay):
delay = min(delay*2, 30*time.Second)
case <-ctx.Done():
return ctx.Err()
}
}
}
}
return fmt.Errorf("command failed after %d attempts: %w", maxAttempts, lastErr)
}
Running Commands in Parallel
For independent commands that can run concurrently — parallel builds, batch downloads, fan-out operations:
type CommandResult struct {
Name string
Out string
Err error
}
func runParallel(ctx context.Context, cmds [][]string) []CommandResult {
results := make([]CommandResult, len(cmds))
var wg sync.WaitGroup
for i, args := range cmds {
wg.Add(1)
go func(idx int, cmdArgs []string) {
defer wg.Done()
out, err := exec.CommandContext(ctx, cmdArgs[0], cmdArgs[1:]...).CombinedOutput()
results[idx] = CommandResult{
Name: cmdArgs[0],
Out: string(out),
Err: err,
}
}(i, args)
}
wg.Wait()
return results
}
// Usage
results := runParallel(ctx, [][]string{
{"go", "vet", "./..."},
{"golangci-lint", "run"},
{"go", "test", "-race", "./..."},
})
for _, r := range results {
if r.Err != nil {
fmt.Printf("FAILED %s: %v\n%s\n", r.Name, r.Err, r.Out)
}
}
Practical: Build System Example
A typical build pipeline that wraps multiple tools:
type Builder struct {
ctx context.Context
workDir string
env []string
}
func (b *Builder) run(name string, args ...string) error {
cmd := exec.CommandContext(b.ctx, name, args...)
cmd.Dir = b.workDir
cmd.Env = b.env
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
slog.Info("running", slog.String("cmd", name), slog.Any("args", args))
return cmd.Run()
}
func (b *Builder) Build(version string) error {
steps := []struct {
name string
fn func() error
}{
{"test", func() error { return b.run("go", "test", "./...") }},
{"vet", func() error { return b.run("go", "vet", "./...") }},
{"build", func() error {
return b.run("go", "build",
"-ldflags", fmt.Sprintf("-s -w -X main.version=%s", version),
"-o", "dist/myapp", "./cmd/myapp")
}},
}
for _, step := range steps {
slog.Info("build step", slog.String("step", step.name))
if err := step.fn(); err != nil {
return fmt.Errorf("step %s: %w", step.name, err)
}
}
return nil
}
Summary
exec.CommandContextwith a context timeout is mandatory — external commands can hang indefinitelycmd.Output()for small captured output;cmd.Stdout = os.Stdoutfor streaming long-running commandsexec.ExitError.ExitCode()gives the exit status;ExitError.Stderrhas the error output- Forward OS signals to child processes explicitly — they don’t propagate automatically
- Use
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}to signal whole process groups - Retry transient failures with exponential backoff; always check
ctx.Err()between retries
Comments