Go makes a compelling replacement for shell scripts. It has static typing, real error handling, cross-platform support, and produces a single statically-linked binary — no interpreter required. At the same time, Go interoperates naturally with shell tools through the os/exec package when you need to call external programs.
This guide covers both directions: calling shell commands from Go code, and writing Go programs that behave like shell scripts. For CLI framework context see building CLIs with Cobra and command-line flag parsing.
The os/exec Package
The foundation for all subprocess work in Go is exec.Command. It creates a *exec.Cmd describing what to run — it does not execute anything yet. You then configure the command and call Run, Output, or Start.
The most important thing to understand upfront: never pass user input through a shell interpreter. The safe pattern is always exec.Command("program", "arg1", "arg2") with arguments as separate strings, not exec.Command("sh", "-c", "program "+userInput). The separate-argument form is not vulnerable to shell injection because no shell interprets the arguments.
Running a Command and Capturing Output
For the common case of “run this and give me the output”:
package main
import (
"fmt"
"os/exec"
"strings"
)
func main() {
// exec.Command takes the binary and each argument separately — no shell expansion
out, err := exec.Command("git", "rev-parse", "--short", "HEAD").Output()
if err != nil {
// err is *exec.ExitError when the command ran but exited non-zero
fmt.Println("error:", err)
return
}
commit := strings.TrimSpace(string(out))
fmt.Println("Current commit:", commit)
}
Output() captures stdout and returns it. If you also need stderr — for example to include error messages in logs — use CombinedOutput() instead.
When a command exits with a non-zero status, err will be an *exec.ExitError. You can inspect the exit code:
out, err := exec.Command("diff", "a.txt", "b.txt").Output()
if err != nil {
var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
fmt.Println("exit code:", exitErr.ExitCode())
fmt.Println("stderr:", string(exitErr.Stderr))
}
return
}
diff exits 1 when files differ (not an error in the “program failed” sense) and 2 on a real error. Checking the exit code lets you handle these cases differently.
Streaming Output in Real Time
Output() waits for the command to finish before returning all output at once. For long-running commands — builds, test suites, log tailing — you want to stream output as it arrives. Connect cmd.Stdout and cmd.Stderr directly to your writers:
func runWithLiveOutput(name string, args ...string) error {
cmd := exec.Command(name, args...)
cmd.Stdout = os.Stdout // stream directly to terminal
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return fmt.Errorf("%s: %w", name, err)
}
return nil
}
// Usage
if err := runWithLiveOutput("go", "test", "./...", "-v"); err != nil {
log.Fatal(err)
}
You can write to any io.Writer — a log file, a buffer, a network connection. If you want to capture output while also showing it in the terminal, use io.MultiWriter:
var buf bytes.Buffer
cmd.Stdout = io.MultiWriter(os.Stdout, &buf)
cmd.Run()
// buf now contains everything that was printed
Context and Timeouts
Any subprocess that talks to the network or does file I/O can hang. Always apply a timeout for non-interactive commands using exec.CommandContext:
func runWithTimeout(timeout time.Duration, name string, args ...string) (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
out, err := exec.CommandContext(ctx, name, args...).Output()
if err != nil {
if ctx.Err() == context.DeadlineExceeded {
return "", fmt.Errorf("command timed out after %s: %s %s", timeout, name, strings.Join(args, " "))
}
return "", err
}
return strings.TrimSpace(string(out)), nil
}
When the context is cancelled or deadline exceeded, CommandContext sends SIGKILL to the process and Run/Output returns immediately. The error will be a context.DeadlineExceeded wrapped in an *exec.ExitError.
Working Directory and Environment
By default, a subprocess inherits the current working directory and environment of the parent process. You can override both:
cmd := exec.Command("make", "build")
// Run in a specific directory
cmd.Dir = "/path/to/project"
// Start with a clean environment (only what you set explicitly)
cmd.Env = []string{
"HOME=/tmp",
"PATH=/usr/local/bin:/usr/bin:/bin",
"GOOS=linux",
"GOARCH=amd64",
}
// Or inherit parent env and add/override specific variables
cmd.Env = append(os.Environ(),
"CGO_ENABLED=0",
"GOFLAGS=-trimpath",
)
out, err := cmd.Output()
The clean environment approach (cmd.Env = []string{...}) is useful for reproducible builds where you don’t want the developer’s shell customizations to affect the output. The inherited approach is more convenient when you want most of the parent’s environment but need to override a few values.
Piping Between Commands
To chain commands like ps aux | grep go | wc -l, use cmd.StdoutPipe() to connect one command’s stdout to another’s stdin:
func countGoProcesses() (int, error) {
ps := exec.Command("ps", "aux")
grep := exec.Command("grep", "[g]o")
// Connect ps stdout → grep stdin
var err error
grep.Stdin, err = ps.StdoutPipe()
if err != nil {
return 0, err
}
var grepOut bytes.Buffer
grep.Stdout = &grepOut
if err := ps.Start(); err != nil {
return 0, fmt.Errorf("ps: %w", err)
}
if err := grep.Start(); err != nil {
return 0, fmt.Errorf("grep: %w", err)
}
// Wait in order: ps first, then grep reads the remaining data
if err := ps.Wait(); err != nil {
return 0, fmt.Errorf("ps wait: %w", err)
}
if err := grep.Wait(); err != nil {
// grep exits 1 when no lines match — that's not a real error here
var exitErr *exec.ExitError
if errors.As(err, &exitErr) && exitErr.ExitCode() == 1 {
return 0, nil
}
return 0, fmt.Errorf("grep wait: %w", err)
}
return strings.Count(grepOut.String(), "\n"), nil
}
For more than two commands in a chain, this gets unwieldy quickly. A cleaner alternative: run each command separately, passing the previous command’s output as a bytes.Reader to the next command’s Stdin. You trade parallelism for simplicity, which is almost always the right trade for scripting tasks.
Writing Go Programs That Act Like Shell Scripts
Go replaces shell scripts best for tasks that involve:
- Non-trivial error handling logic
- Parallel execution (downloads, builds, transformations)
- Platform portability (your script runs on Linux, macOS, and Windows)
- Complex data transformations
Here’s a Go program that mirrors what a shell script might do — find all Go files modified in the last 24 hours and run gofmt -l on them:
package main
import (
"fmt"
"io/fs"
"os"
"os/exec"
"path/filepath"
"time"
)
func main() {
cutoff := time.Now().Add(-24 * time.Hour)
var recent []string
// filepath.WalkDir is the idiomatic Go equivalent of `find`
err := filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && d.Name() == "vendor" {
return filepath.SkipDir // skip vendor directory
}
if filepath.Ext(path) != ".go" {
return nil
}
info, err := d.Info()
if err != nil {
return err
}
if info.ModTime().After(cutoff) {
recent = append(recent, path)
}
return nil
})
if err != nil {
fmt.Fprintln(os.Stderr, "walk error:", err)
os.Exit(1)
}
if len(recent) == 0 {
fmt.Println("no recently modified Go files")
return
}
// Run gofmt on each file
args := append([]string{"-l"}, recent...)
out, err := exec.Command("gofmt", args...).Output()
if err != nil {
fmt.Fprintln(os.Stderr, "gofmt error:", err)
os.Exit(1)
}
if len(out) == 0 {
fmt.Println("all files formatted correctly")
return
}
fmt.Println("unformatted files:")
fmt.Print(string(out))
os.Exit(1)
}
This is more verbose than the equivalent two-line shell script, but it handles errors precisely, works identically on all platforms, and is easy to extend — add parallel execution, JSON output, or filtering by module without reaching for obscure shell syntax.
Cross-Platform Considerations
Shell commands are not portable. ls is dir on Windows. Path separators differ. Line endings differ.
When you need to run external commands cross-platform, use build tags or runtime detection:
func listDir(path string) ([]string, error) {
var cmd *exec.Cmd
if runtime.GOOS == "windows" {
cmd = exec.Command("cmd", "/C", "dir", "/B", path)
} else {
cmd = exec.Command("ls", "-1", path)
}
out, err := cmd.Output()
if err != nil {
return nil, err
}
lines := strings.Split(strings.TrimSpace(string(out)), "\n")
return lines, nil
}
In most cases, the better answer is to use Go’s standard library directly instead of shelling out: os.ReadDir replaces ls, os.Remove replaces rm, filepath.WalkDir replaces find. Using the standard library avoids platform differences entirely and usually produces cleaner error messages.
Shell Injection: The One Rule That Matters
Never construct a command string from user input and pass it to a shell:
// ❌ Shell injection — user can pass "foo; rm -rf /"
userInput := r.URL.Query().Get("filename")
exec.Command("sh", "-c", "cat "+userInput).Run()
// ✅ Safe — shell never sees the input, no injection possible
exec.Command("cat", userInput).Run()
When you pass arguments as separate strings to exec.Command, they are passed directly to execve (on Unix) without any shell processing. No quoting, no globbing, no command substitution — the argument is what you wrote, literally.
If you genuinely need a shell feature (glob expansion, pipes in a single string), sanitize carefully and consider whether the design can be restructured to avoid it. For most tasks, the explicit exec.Command with separate arguments is both safer and clearer.
Executing Script Files
If you have existing shell scripts and need to invoke them from Go:
func runScript(ctx context.Context, scriptPath string, args ...string) error {
// Verify the script exists before attempting to run it
if _, err := os.Stat(scriptPath); err != nil {
return fmt.Errorf("script not found: %w", err)
}
// On Unix, ensure it's executable — os.Chmod is a no-op on Windows
if runtime.GOOS != "windows" {
if err := os.Chmod(scriptPath, 0755); err != nil {
return fmt.Errorf("chmod: %w", err)
}
}
cmd := exec.CommandContext(ctx, scriptPath, args...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
On Windows, .sh scripts won’t run directly — use WSL, Git Bash, or convert critical scripts to Go. Alternatively, detect the platform and choose the right interpreter:
func runShellScript(ctx context.Context, script string, args ...string) error {
var cmd *exec.Cmd
if runtime.GOOS == "windows" {
allArgs := append([]string{script}, args...)
cmd = exec.CommandContext(ctx, "bash", allArgs...) // requires Git Bash or WSL
} else {
allArgs := append([]string{script}, args...)
cmd = exec.CommandContext(ctx, "/bin/sh", allArgs...)
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}
Common Mistakes
Calling cmd.Run() after cmd.Output(). Each Cmd can only be run once. Output() already calls Run internally. Create a new exec.Command(...) for each execution.
Not waiting for the process. If you call cmd.Start() without a matching cmd.Wait(), the child process becomes a zombie on Unix and leaks resources. Use cmd.Run() (which calls both) unless you need to do work concurrently with the subprocess.
Using CombinedOutput() when you need separate streams. CombinedOutput mixes stdout and stderr into one byte slice. If you need to parse stdout while logging stderr separately, set cmd.Stdout and cmd.Stderr to different writers and call cmd.Run().
Forgetting that cmd.Wait() closes the stdout pipe. If you use cmd.StdoutPipe(), you must read all output from the pipe before calling cmd.Wait() — or start a goroutine that reads from it. Failing to do so causes a deadlock when the pipe buffer fills.
Summary
- Use
exec.Command("binary", "arg1", "arg2")with separate arguments — never build shell command strings from user input Output()for captured output,cmd.Stdout = os.Stdoutfor streaming,CombinedOutput()when you need both streams together- Always apply
context.WithTimeoutfor commands that could hang - Prefer Go’s standard library (
os.ReadDir,filepath.WalkDir,os.Remove) over shelling out when an equivalent exists — it’s portable and produces better errors - Use
cmd.Dirandcmd.Envto control the subprocess environment explicitly
Resources
- os/exec package documentation
- OWASP: Command Injection
- Go by Example: Spawning Processes
- Go by Example: Execing Processes
Comments