Text processing is one of the most common tasks in real applications — parsing logs, validating input, transforming data, searching content. Go’s strings package covers most day-to-day needs with simple, well-named functions. For pattern matching, regexp handles the rest. And for performance-critical string building, understanding when to use strings.Builder vs concatenation vs fmt.Sprintf matters more than it seems.
This guide covers the practical toolkit: the strings package, Unicode-aware operations, efficient string building, and a handful of useful text algorithms with their real-world applications.
For related topics see Go bytes and runes, Go regular expressions, and Go standard library fmt and strings.
The strings Package: Your First Stop
The strings package covers searching, replacing, splitting, joining, and case conversion. These functions operate on UTF-8 strings and are the right starting point for almost any text task:
s := " Hello, World! "
strings.TrimSpace(s) // "Hello, World!"
strings.ToLower(s) // " hello, world! "
strings.Contains(s, "World") // true
strings.HasPrefix(s, " Hello") // true
strings.Index(s, "World") // 9
strings.Count(s, "l") // 3
strings.Replace(s, "World", "Go", 1) // " Hello, Go! "
strings.ReplaceAll(s, "l", "L") // " HeLLo, WorLd! "
For splitting and joining — the bread and butter of CSV-like parsing and URL manipulation:
parts := strings.Split("a,b,c", ",") // ["a", "b", "c"]
strings.Join(parts, " | ") // "a | b | c"
strings.Fields(" hello world ") // ["hello", "world"] — splits on any whitespace
strings.Fields is more useful than strings.Split(s, " ") when input has inconsistent whitespace — it treats any run of whitespace as a delimiter.
When strings.Cut Is Cleaner Than Split
Go 1.18 added strings.Cut, which is perfect for splitting a string on the first occurrence of a separator:
// Parse "key=value" pairs
line := "Content-Type: application/json; charset=utf-8"
key, value, found := strings.Cut(line, ": ")
// key="Content-Type", value="application/json; charset=utf-8", found=true
Cut is cleaner than strings.SplitN(s, sep, 2) for this common pattern — it’s more readable and returns a clear found boolean.
Efficient String Building
String concatenation with + in a loop creates a new string on every iteration — each intermediate result is a separate allocation. For building strings incrementally, use strings.Builder:
// ❌ O(n²) allocations — each + creates a new string
result := ""
for _, word := range words {
result += word + " "
}
// ✅ O(n) — Builder amortizes allocations like a dynamic array
var b strings.Builder
b.Grow(estimatedSize) // optional: pre-allocate if you know the size
for _, word := range words {
b.WriteString(word)
b.WriteByte(' ')
}
result := b.String()
strings.Builder works like bytes.Buffer but is typed for string output. Pre-allocating with b.Grow(n) avoids reallocation entirely when you know the approximate output size.
For templated output where you’re combining fixed strings and variable values, fmt.Sprintf is fine for occasional use — it’s readable and flexible. But in hot loops, the format string parsing has overhead. In benchmarks, strings.Builder with explicit writes is typically 2–5x faster than fmt.Sprintf for the same output.
Unicode-Aware String Processing
A Go string is a sequence of bytes (UTF-8), not a sequence of characters. len(s) returns the byte count, not the character count. For most ASCII text this doesn’t matter, but for international text it does:
s := "Hello, 世界"
fmt.Println(len(s)) // 13 bytes (each Chinese character is 3 bytes in UTF-8)
fmt.Println(len([]rune(s))) // 9 characters (rune = Unicode code point)
The unicode package provides character-level predicates. Use range to iterate by rune rather than byte:
func countVowels(s string) int {
count := 0
for _, r := range s { // r is a rune (int32 Unicode code point)
switch unicode.ToLower(r) {
case 'a', 'e', 'i', 'o', 'u':
count++
}
}
return count
}
For transformations like “remove all non-letter characters”, strings.Map is the idiomatic approach — it applies a function to every rune and filters out the ones where the function returns -1:
func lettersOnly(s string) string {
return strings.Map(func(r rune) rune {
if unicode.IsLetter(r) {
return r
}
return -1 // drop this rune
}, s)
}
lettersOnly("Hello, World! 123") // "HelloWorld"
Word Frequency Counter
A practical algorithm for search relevance scoring, text analysis, and autocomplete. The key detail is normalization — lowercasing and stripping punctuation before counting:
func wordFrequency(text string) map[string]int {
freq := make(map[string]int)
// Trim punctuation attached to words, then split on whitespace
for _, word := range strings.Fields(text) {
word = strings.ToLower(strings.Trim(word, `.,!?;:'"()-`))
if word != "" {
freq[word]++
}
}
return freq
}
For top-N most frequent words, pair this with a sort:
type wordCount struct { word string; count int }
func topN(freq map[string]int, n int) []wordCount {
wcs := make([]wordCount, 0, len(freq))
for word, count := range freq {
wcs = append(wcs, wordCount{word, count})
}
sort.Slice(wcs, func(i, j int) bool {
return wcs[i].count > wcs[j].count
})
if n < len(wcs) {
return wcs[:n]
}
return wcs
}
Levenshtein Distance: Measuring String Similarity
Levenshtein distance counts the minimum number of single-character edits (insertions, deletions, substitutions) needed to transform one string into another. It’s the algorithm behind spell checkers, fuzzy search, and “did you mean?” suggestions.
The dynamic programming solution fills an (m+1) × (n+1) table where dp[i][j] is the edit distance between the first i characters of s1 and the first j characters of s2:
func levenshtein(s1, s2 string) int {
r1, r2 := []rune(s1), []rune(s2)
m, n := len(r1), len(r2)
dp := make([][]int, m+1)
for i := range dp {
dp[i] = make([]int, n+1)
dp[i][0] = i // deleting i chars from s1
}
for j := range dp[0] {
dp[0][j] = j // inserting j chars to reach s2
}
for i := 1; i <= m; i++ {
for j := 1; j <= n; j++ {
if r1[i-1] == r2[j-1] {
dp[i][j] = dp[i-1][j-1] // no edit needed
} else {
dp[i][j] = 1 + min(dp[i-1][j], dp[i][j-1], dp[i-1][j-1])
// delete insert substitute
}
}
}
return dp[m][n]
}
Using rune slices ([]rune) instead of bytes ensures the distance is measured in characters, not bytes — important for any non-ASCII text.
Real-world use: to implement a “did you mean?” suggestion, compute the Levenshtein distance between the query and every word in your dictionary, and return those within distance 1 or 2. For large dictionaries, use a BK-tree to avoid comparing against every word.
Longest Common Substring
The longest common substring finds the longest sequence of characters that appears contiguously in both strings. This is different from longest common subsequence (which allows gaps). The DP approach is similar to Levenshtein:
func longestCommonSubstring(s1, s2 string) string {
r1, r2 := []rune(s1), []rune(s2)
m, n := len(r1), len(r2)
dp := make([][]int, m+1)
for i := range dp {
dp[i] = make([]int, n+1)
}
maxLen, endPos := 0, 0
for i := 1; i <= m; i++ {
for j := 1; j <= n; j++ {
if r1[i-1] == r2[j-1] {
dp[i][j] = dp[i-1][j-1] + 1
if dp[i][j] > maxLen {
maxLen = dp[i][j]
endPos = i
}
}
}
}
return string(r1[endPos-maxLen : endPos])
}
Applications: detecting plagiarism, DNA sequence alignment, diff tools.
Tokenization with Regex
When splitting on simple delimiters isn’t enough — for example, tokenizing code or natural language — regexp handles complex patterns:
// Tokenize a string into words, numbers, and punctuation
var tokenRe = regexp.MustCompile(`\w+|[^\w\s]`)
func tokenize(s string) []string {
return tokenRe.FindAllString(s, -1)
}
tokenize("Hello, world! 42 items.")
// ["Hello", "world", "42", "items"]
// (punctuation like "," and "!" matched by [^\w\s], filtered or kept as needed)
For performance-critical tokenization, compile the regex once at package level with regexp.MustCompile (which panics at startup if the pattern is invalid — appropriate for hardcoded patterns). Never compile a regex inside a loop.
Performance Notes
String operations that look cheap can be expensive at scale:
strings.Containsin a loop: if you’re searching a fixed set of substrings, build amap[string]boollookup instead.strings.Splitthenstrings.Join: if you’re only modifying some fields, this creates many allocations. Considerstrings.Builderwith manual iteration.regexpin a loop: always compile once, reuse the*regexp.Regexp. Compilation is expensive; matching is fast.- Converting between
stringand[]byte: each conversion allocates. If you’re reading from anio.Readerand doing string operations, consider usingbytes.Bufferand thebytespackage to stay in byte-slice land.
Use go test -bench and go test -benchmem to measure before optimizing — string operation bottlenecks are often not where you expect them.
Summary
- The
stringspackage covers searching, splitting, replacing, and case conversion — reach for it before rolling your own - Use
strings.Builderfor string construction in loops; pre-allocate withGrowwhen size is known - Iterate with
rangeto work on runes (characters), not bytes — essential for Unicode correctness strings.Mapis the idiomatic way to filter or transform rune-by-rune- Levenshtein distance powers spell check and fuzzy matching; the DP solution is O(m×n) time and space
- Compile
regexppatterns once at package level, never inside loops
Resources
- strings package documentation
- unicode package documentation
- regexp package documentation
- Go Strings, bytes, runes and characters (Go Blog)
Comments