Skip to main content

AST Manipulation and Analysis in Go

Published: May 8, 2026 • Updated: August 29, 2026 Larry Qu 11 min read
Table of Contents

Go is one of the few languages where the standard library ships everything you need to parse, walk, and rewrite Go source code itself. The go/parser, go/ast, go/types, and go/format packages form a complete toolkit for building static analysis tools, linters, code generators, and automated refactoring utilities — the same packages that power gofmt, gopls, and go vet.

This guide covers the full pipeline: parsing source into an AST, traversing and querying the tree, building a simple linter, and generating new code. For broader context see Go code generation and Go best practices.

What Is an AST?

When Go compiles your source code, the first step is turning text into a structured tree where every language construct — a function declaration, a binary expression, a variable assignment — becomes a node with typed fields pointing to its children.

For a simple assignment like x := 5 + 3, the tree looks like this:

AssignStmt
├── Lhs: []Expr
│   └── Ident { Name: "x" }
├── Tok: :=
└── Rhs: []Expr
    └── BinaryExpr
        ├── X: BasicLit { Kind: INT, Value: "5" }
        ├── Op: +
        └── Y: BasicLit { Kind: INT, Value: "3" }

Every node in Go’s AST implements ast.Node, which provides Pos() and End() — byte offsets into the source file. You use a token.FileSet to translate those offsets into human-readable file/line/column positions.

The reason to work with ASTs rather than regex on source text: the tree captures structure that text patterns cannot. You can reliably find “every function whose first parameter is a context.Context” or “every call to os.Open that doesn’t check the error” — questions that would require fragile, incomplete regex.

The Three Core Packages

Package Purpose
go/parser Turns source text into *ast.File
go/ast Defines all node types; provides Walk and Inspect
go/token Defines token kinds and the FileSet for position tracking
go/format Prints AST nodes back to correctly formatted Go source
go/types Type-checks an AST; resolves what each identifier refers to

For most tooling tasks you’ll use all five together.

Parsing Source Code

The entry point is parser.ParseFile. It accepts either a filename (reads from disk) or an explicit source string, and returns an *ast.File representing one Go file:

package main

import (
    "fmt"
    "go/ast"
    "go/parser"
    "go/token"
)

func main() {
    src := `
package main

import "fmt"

func greet(name string) string {
    return fmt.Sprintf("Hello, %s!", name)
}
`
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "example.go", src, parser.AllErrors|parser.ParseComments)
    if err != nil {
        fmt.Println("parse error:", err)
        return
    }

    fmt.Println("Package:", file.Name.Name)
    fmt.Println("Declarations:", len(file.Decls))
}

A few things worth noting:

  • The token.FileSet is shared across all files you parse together. It maps byte offsets to positions. Always create one FileSet per analysis run, not one per file.
  • parser.AllErrors makes the parser report every error rather than stopping at the first one — important for linters that need to see partial trees.
  • parser.ParseComments retains comment nodes in the tree. Without it, //go:generate directives and doc comments are stripped.

To parse an entire package directory at once:

pkgs, err := parser.ParseDir(fset, "./mypkg", nil, parser.AllErrors)
if err != nil {
    log.Fatal(err)
}
for name, pkg := range pkgs {
    fmt.Printf("Package %s has %d files\n", name, len(pkg.Files))
}

Traversing the Tree

ast.Inspect — The Simple Case

ast.Inspect walks every node depth-first, calling your function on the way in. Return true to recurse into children, false to skip the subtree:

// Find all function declarations and print their names and parameter counts
ast.Inspect(file, func(n ast.Node) bool {
    fn, ok := n.(*ast.FuncDecl)
    if !ok {
        return true // not a function, keep walking
    }

    params := 0
    if fn.Type.Params != nil {
        params = fn.Type.Params.NumFields()
    }
    pos := fset.Position(fn.Pos())
    fmt.Printf("%s:%d: func %s (%d params)\n", pos.Filename, pos.Line, fn.Name.Name, params)
    return true
})

ast.Inspect is the right tool for most read-only analysis. It’s concise and handles all node types without requiring you to implement an interface.

ast.Walk — The Visitor Pattern

When your analysis has state that grows as you traverse (collecting declarations, tracking scope), implement ast.Visitor:

// Collect all identifiers that start with a capital letter (exported names)
type ExportedFinder struct {
    names []string
}

func (v *ExportedFinder) Visit(node ast.Node) ast.Visitor {
    ident, ok := node.(*ast.Ident)
    if ok && ident.IsExported() {
        v.names = append(v.names, ident.Name)
    }
    return v // return v to continue walking, nil to stop recursing
}

finder := &ExportedFinder{}
ast.Walk(finder, file)
fmt.Println("Exported names:", finder.names)

Visit is called on every node. Returning v (the same visitor) continues traversal into children. Returning nil prunes the subtree — useful for skipping function bodies when you only care about top-level declarations.

Querying Patterns Across a File

A common task is finding all calls to a particular function. For example, finding every place os.Open is called:

func findOsOpenCalls(file *ast.File) []token.Pos {
    var positions []token.Pos

    ast.Inspect(file, func(n ast.Node) bool {
        call, ok := n.(*ast.CallExpr)
        if !ok {
            return true
        }

        // A selector expression looks like: os.Open
        sel, ok := call.Fun.(*ast.SelectorExpr)
        if !ok {
            return true
        }

        pkg, ok := sel.X.(*ast.Ident)
        if ok && pkg.Name == "os" && sel.Sel.Name == "Open" {
            positions = append(positions, call.Pos())
        }
        return true
    })

    return positions
}

This works for simple cases. For reliable cross-package analysis (distinguishing your os.Open from a local os variable that shadows the package), you need type information from go/types. But for single-file analysis or trusted source, the AST alone is often sufficient.

Building a Simple Linter

Let’s build a linter that reports functions with more than one return value where the last return value is not an error. This is a common convention check.

package main

import (
    "fmt"
    "go/ast"
    "go/parser"
    "go/token"
)

type Issue struct {
    Pos     token.Position
    Message string
}

func checkErrorLast(fset *token.FileSet, file *ast.File) []Issue {
    var issues []Issue

    ast.Inspect(file, func(n ast.Node) bool {
        fn, ok := n.(*ast.FuncDecl)
        if !ok {
            return true
        }

        results := fn.Type.Results
        if results == nil || results.NumFields() < 2 {
            return true // single return value or void — skip
        }

        // Get the last result type
        last := results.List[len(results.List)-1]
        ident, ok := last.Type.(*ast.Ident)
        if !ok {
            return true // not a simple type name (could be *SomeError — acceptable)
        }

        if ident.Name != "error" {
            issues = append(issues, Issue{
                Pos:     fset.Position(fn.Pos()),
                Message: fmt.Sprintf("function %q has multiple returns but last is %q, not error", fn.Name.Name, ident.Name),
            })
        }
        return true
    })

    return issues
}

func main() {
    src := `
package main

// Good: error is last
func ReadFile(name string) ([]byte, error) { return nil, nil }

// Flagged: multiple returns, last is not error
func GetUserAndCode(id int) (error, int) { return nil, 0 }
`
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "check.go", src, parser.AllErrors)
    if err != nil {
        fmt.Println("parse error:", err)
        return
    }

    issues := checkErrorLast(fset, file)
    for _, issue := range issues {
        fmt.Printf("%s: %s\n", issue.Pos, issue.Message)
    }
}

Running this outputs:

check.go:8:1: function "GetUserAndCode" has multiple returns but last is "int", not error

For production linters, the go/analysis package (part of the standard library as of Go 1.12) provides a structured framework with proper type checking, facts propagation across packages, and integration with go vet and IDEs. The pattern above is a good starting point for understanding; go/analysis is the right foundation for anything you want to ship.

Cyclomatic Complexity: Counting Decision Points

Cyclomatic complexity measures how many independent paths through a function exist — each if, for, switch case, and &&/|| adds one. Functions above ~15 are difficult to test comprehensively.

// countComplexity returns the cyclomatic complexity of a function body.
// Baseline is 1; each branch adds 1.
func countComplexity(body *ast.BlockStmt) int {
    complexity := 1

    ast.Inspect(body, func(n ast.Node) bool {
        switch n.(type) {
        case *ast.IfStmt:
            complexity++
        case *ast.ForStmt, *ast.RangeStmt:
            complexity++
        case *ast.CaseClause: // each case in a switch
            complexity++
        case *ast.CommClause: // each case in a select
            complexity++
        case *ast.BinaryExpr:
            // && and || introduce additional paths
            be := n.(*ast.BinaryExpr)
            if be.Op.String() == "&&" || be.Op.String() == "||" {
                complexity++
            }
        }
        return true
    })

    return complexity
}

Combined with the function finder above, you can flag all functions over a threshold — a useful gate in CI pipelines.

Modifying the AST and Formatting Output

The AST is a live data structure — you can modify nodes directly, then print the result back to source using go/format. This is how automated refactoring tools work.

Example: rename all uses of an identifier within a file.

import (
    "bytes"
    "go/ast"
    "go/format"
    "go/parser"
    "go/token"
)

func renameIdent(src, oldName, newName string) (string, error) {
    fset := token.NewFileSet()
    file, err := parser.ParseFile(fset, "in.go", src, 0)
    if err != nil {
        return "", err
    }

    ast.Inspect(file, func(n ast.Node) bool {
        ident, ok := n.(*ast.Ident)
        if ok && ident.Name == oldName {
            ident.Name = newName
        }
        return true
    })

    var buf bytes.Buffer
    if err := format.Node(&buf, fset, file); err != nil {
        return "", err
    }
    return buf.String(), nil
}

format.Node runs the same formatter as gofmt, so the output is always correctly indented and spaced regardless of what you changed in the tree. This is a major advantage over string manipulation — you can’t accidentally produce malformed Go.

For more complex rewrites involving type-aware transformations, look at golang.org/x/tools/go/analysis and golang.org/x/tools/go/ast/astutil. The astutil.Apply function provides a cleaner API for tree rewriting with pre- and post-order callbacks.

Generating Code from AST Nodes

Code generation in Go is typically done by building an AST programmatically and then printing it, or more commonly, by rendering text templates. The AST approach guarantees valid Go output:

// Build a simple function declaration: func Add(a, b int) int { return a + b }
func buildAddFunc() *ast.FuncDecl {
    return &ast.FuncDecl{
        Name: ast.NewIdent("Add"),
        Type: &ast.FuncType{
            Params: &ast.FieldList{
                List: []*ast.Field{
                    {
                        Names: []*ast.Ident{ast.NewIdent("a"), ast.NewIdent("b")},
                        Type:  ast.NewIdent("int"),
                    },
                },
            },
            Results: &ast.FieldList{
                List: []*ast.Field{{Type: ast.NewIdent("int")}},
            },
        },
        Body: &ast.BlockStmt{
            List: []ast.Stmt{
                &ast.ReturnStmt{
                    Results: []ast.Expr{
                        &ast.BinaryExpr{
                            X:  ast.NewIdent("a"),
                            Op: token.ADD,
                            Y:  ast.NewIdent("b"),
                        },
                    },
                },
            },
        },
    }
}

Building AST nodes manually is verbose. In practice, most Go code generators use text/template to produce source text, then pass it through format.Source to validate and format it:

import (
    "bytes"
    "go/format"
    "text/template"
)

const funcTmpl = `
package {{.Package}}

func {{.Name}}({{range .Params}}{{.Name}} {{.Type}}, {{end}}) {{.ReturnType}} {
    // TODO: implement
}
`

func generateFunc(data interface{}) (string, error) {
    t := template.Must(template.New("func").Parse(funcTmpl))
    var buf bytes.Buffer
    if err := t.Execute(&buf, data); err != nil {
        return "", err
    }
    // format.Source validates and formats the generated code
    formatted, err := format.Source(buf.Bytes())
    if err != nil {
        return "", fmt.Errorf("generated invalid Go: %w\nsource:\n%s", err, buf.String())
    }
    return string(formatted), nil
}

If format.Source returns an error, your template produced invalid Go. The error message tells you where, which is far easier to debug than a mysterious compile failure later.

The go/analysis Framework for Production Linters

For linters you want to run via go vet or integrate into editors, use go/analysis:

import "golang.org/x/tools/go/analysis"

var Analyzer = &analysis.Analyzer{
    Name: "errcheck",
    Doc:  "check that error returns are not ignored",
    Run:  run,
}

func run(pass *analysis.Pass) (interface{}, error) {
    for _, file := range pass.Files {
        ast.Inspect(file, func(n ast.Node) bool {
            // ExprStmt wraps a standalone function call
            exprStmt, ok := n.(*ast.ExprStmt)
            if !ok {
                return true
            }
            call, ok := exprStmt.X.(*ast.CallExpr)
            if !ok {
                return true
            }

            // Check if the call returns an error that's being discarded
            sig, ok := pass.TypesInfo.TypeOf(call.Fun).(*types.Signature)
            if !ok {
                return true
            }
            results := sig.Results()
            if results.Len() > 0 {
                last := results.At(results.Len() - 1)
                if last.Type().String() == "error" {
                    pass.Reportf(call.Pos(), "error return value of %s is not checked", call.Fun)
                }
            }
            return true
        })
    }
    return nil, nil
}

The pass object gives you the parsed AST, type information (pass.TypesInfo), and a Reportf method to emit diagnostics at the right source position. The framework handles multi-package analysis, caching, and IDE integration automatically.

Common Pitfalls

Forgetting that one FileSet spans all files. If you create a new token.FileSet for each file, positions from different files become incomparable and fset.Position() returns wrong results. Always pass the same fset when parsing multiple files in one analysis run.

Ignoring partial parse errors. parser.ParseFile with parser.AllErrors can return both a non-nil *ast.File and a non-nil error. The file is a partial tree — useful for analysis, but some nodes may be nil. Always handle the error and guard against nil nodes.

Modifying the AST while walking it. ast.Inspect and ast.Walk do not support concurrent modification. Build a list of changes during traversal, then apply them after the walk completes.

Confusing AST identity with source position. Two *ast.Ident nodes with the same Name are distinct nodes — they represent different occurrences in source. If you need to know whether two identifiers refer to the same declaration, use go/types type information (TypesInfo.ObjectOf), not name comparison.

Summary

  • go/parser + go/ast + go/token give you a complete pipeline from source text to structured tree and back
  • Use ast.Inspect for simple stateless traversals; implement ast.Visitor when you need to carry state through the walk
  • format.Node and format.Source print AST nodes back to correctly formatted Go — essential for code generation and rewriting
  • For production linters that need type information, use golang.org/x/tools/go/analysis — it integrates with go vet, editors, and CI
  • Build code generators with text/template + format.Source rather than raw AST construction — it’s far less verbose and equally safe

Resources

Comments

Share this article

Twitter Bluesky Mastodon LinkedIn Facebook Reddit Hacker News WhatsApp Telegram Email

Scan to read on mobile

👍 Was this article helpful?