Semaphores and Rate Limiting in Go

Two of the most common causes of production outages are uncontrolled concurrency and unbounded request rates. Spawning 10,000 goroutines to handle 10,000 simultaneous requests will consume memory, thrash the scheduler, and starve other processes — even if each goroutine is individually correct. Rate limiting protects both your service and the services it calls.

Go has no built-in semaphore or rate limiter, but both are straightforward to implement using channels. This guide covers the core algorithms, when to use each, and how to reach for golang.org/x/time/rate for production use.

For more context see Go concurrency patterns, Go worker pools, and Go context cancellation.

Semaphores: Bounding Concurrency

A semaphore controls how many goroutines can be doing something at once. The most direct Go implementation uses a buffered channel as a counting semaphore: sending acquires a slot, receiving releases it.

type Semaphore struct {
    sem chan struct{}
}

func NewSemaphore(maxConcurrent int) *Semaphore {
    return &Semaphore{sem: make(chan struct{}, maxConcurrent)}
}

func (s *Semaphore) Acquire() { s.sem <- struct{}{} }
func (s *Semaphore) Release() { <-s.sem }

The channel capacity sets the ceiling. When the channel is full (all slots taken), the next Acquire blocks until someone calls Release. This is exactly the right behavior: callers wait, not crash.

In practice you almost always want context support so goroutines can be cancelled while waiting:

func (s *Semaphore) AcquireCtx(ctx context.Context) error {
    select {
    case s.sem <- struct{}{}:
        return nil
    case <-ctx.Done():
        return ctx.Err()
    }
}

Using a semaphore to limit parallel HTTP calls to an external API:

func fetchAll(ctx context.Context, urls []string) ([]string, error) {
    sem := NewSemaphore(5) // max 5 in-flight at once
    results := make([]string, len(urls))
    errs := make([]error, len(urls))

    var wg sync.WaitGroup
    for i, url := range urls {
        wg.Add(1)
        go func(i int, url string) {
            defer wg.Done()

            if err := sem.AcquireCtx(ctx); err != nil {
                errs[i] = err
                return
            }
            defer sem.Release()

            resp, err := http.Get(url)
            if err != nil {
                errs[i] = err
                return
            }
            defer resp.Body.Close()
            body, _ := io.ReadAll(resp.Body)
            results[i] = string(body)
        }(i, url)
    }
    wg.Wait()

    for _, err := range errs {
        if err != nil {
            return nil, err
        }
    }
    return results, nil
}

Without the semaphore, launching 500 goroutines against an external service simultaneously would likely hit connection limits, trigger rate limits on the remote side, or cause your own memory to spike. Five concurrent requests is usually more than fast enough — the bottleneck is network I/O, not goroutine count.

When to Use a Semaphore vs a Worker Pool

A semaphore lets goroutines be created freely but gates them at a checkpoint. A worker pool creates a fixed number of goroutines that pull from a queue. The right choice depends on the workload:

• Semaphore: each task has variable setup cost, or tasks arrive asynchronously. Simple to add to existing code.
• Worker pool: tasks are homogenous, you want to amortize goroutine creation cost, or you need backpressure on the producer side. See Go worker pools for the full pattern.

The golang.org/x/sync/semaphore Package

For production use, the extended library provides a weighted semaphore that handles goroutine scheduling more efficiently than a bare channel, especially at high concurrency:

import "golang.org/x/sync/semaphore"

// Max 10 concurrent operations, each costing 1 unit
sem := semaphore.NewWeighted(10)

// Acquire 1 unit, release when done
if err := sem.Acquire(ctx, 1); err != nil {
    return err
}
defer sem.Release(1)

The weighted variant is useful when operations have different resource costs — a heavy query might acquire 4 units while a lightweight lookup acquires 1, letting you express capacity in resource terms rather than just concurrency count.

Rate Limiting: Controlling Throughput Over Time

A semaphore bounds how many things happen simultaneously. A rate limiter bounds how many things happen per unit of time. These are different constraints — you might allow 100 concurrent requests but cap at 1,000 per second, or allow only 1 concurrent request but up to 60 per minute.

Token Bucket: Allowing Bursts

The token bucket algorithm is the most common choice for API rate limiting. Tokens accumulate in a bucket at a steady refill rate up to a maximum capacity. Each request consumes a token. When the bucket is empty, requests are rejected (or wait).

The key property is burst tolerance: if no requests arrive for a while, tokens accumulate, and a subsequent burst can be served immediately up to the bucket capacity. This mirrors how most services actually behave — clients are idle, then send a burst.

Go’s standard implementation lives in golang.org/x/time/rate:

import "golang.org/x/time/rate"

// Allow 100 requests/second with a burst capacity of 20
limiter := rate.NewLimiter(rate.Limit(100), 20)

The limiter exposes three usage patterns depending on what you want to do when the limit is reached:

// 1. Allow: non-blocking, returns false if limit exceeded
if !limiter.Allow() {
    http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
    return
}

// 2. Wait: blocks until a token is available or context is cancelled
if err := limiter.Wait(ctx); err != nil {
    return err // context cancelled while waiting
}

// 3. Reserve: get a token in advance, check when it will be ready
r := limiter.Reserve()
time.Sleep(r.Delay()) // wait exactly as long as needed

Allow is right for APIs where you want to reject immediately and let the client retry. Wait is right for background jobs where blocking is acceptable. Reserve is useful for scheduling work without holding a goroutine during the wait.

Per-Client Rate Limiting

A single global limiter protects your service overall but doesn’t prevent one client from consuming the entire budget. For per-client limiting, maintain a map of limiters:

type ClientLimiter struct {
    mu       sync.Mutex
    limiters map[string]*rate.Limiter
    rate     rate.Limit
    burst    int
}

func NewClientLimiter(r rate.Limit, burst int) *ClientLimiter {
    return &ClientLimiter{
        limiters: make(map[string]*rate.Limiter),
        rate:     r,
        burst:    burst,
    }
}

func (cl *ClientLimiter) getLimiter(key string) *rate.Limiter {
    cl.mu.Lock()
    defer cl.mu.Unlock()

    if l, ok := cl.limiters[key]; ok {
        return l
    }
    l := rate.NewLimiter(cl.rate, cl.burst)
    cl.limiters[key] = l
    return l
}

func (cl *ClientLimiter) Allow(key string) bool {
    return cl.getLimiter(key).Allow()
}

Use it as HTTP middleware, keying on IP address or API key:

func RateLimitMiddleware(limiter *ClientLimiter) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            key := r.Header.Get("X-API-Key")
            if key == "" {
                key = r.RemoteAddr
            }
            if !limiter.Allow(key) {
                w.Header().Set("Retry-After", "1")
                http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
                return
            }
            next.ServeHTTP(w, r)
        })
    }
}

In production, this in-memory map needs pruning (stale entries accumulate) and won’t work across multiple instances. For multi-instance deployments, use Redis with a sliding window script or a service like Redis Cell (CL.THROTTLE).

Building a Token Bucket From Scratch

Understanding the algorithm is useful even if you use x/time/rate in production:

type TokenBucket struct {
    mu         sync.Mutex
    tokens     float64
    maxTokens  float64
    refillRate float64 // tokens per second
    lastRefill time.Time
}

func NewTokenBucket(maxTokens, refillRate float64) *TokenBucket {
    return &TokenBucket{
        tokens:     maxTokens,
        maxTokens:  maxTokens,
        refillRate: refillRate,
        lastRefill: time.Now(),
    }
}

func (tb *TokenBucket) Allow(cost float64) bool {
    tb.mu.Lock()
    defer tb.mu.Unlock()

    // Refill tokens based on elapsed time
    now := time.Now()
    elapsed := now.Sub(tb.lastRefill).Seconds()
    tb.tokens = math.Min(tb.maxTokens, tb.tokens+elapsed*tb.refillRate)
    tb.lastRefill = now

    if tb.tokens >= cost {
        tb.tokens -= cost
        return true
    }
    return false
}

The refill happens lazily on each Allow call — no background goroutine needed. The math is simple: multiply elapsed seconds by the refill rate, clamp to the maximum.

Leaky Bucket: Smoothing Output

The leaky bucket algorithm is the opposite of token bucket in one important way: it smooths output to a constant rate and does not allow bursts. Think of it as a queue with a fixed drain rate.

type LeakyBucket struct {
    queue    chan struct{}
    interval time.Duration // time between each allowed request
}

func NewLeakyBucket(capacity int, rate time.Duration) *LeakyBucket {
    lb := &LeakyBucket{
        queue:    make(chan struct{}, capacity),
        interval: rate,
    }
    go lb.drain()
    return lb
}

// drain leaks one item every interval
func (lb *LeakyBucket) drain() {
    ticker := time.NewTicker(lb.interval)
    defer ticker.Stop()
    for range ticker.C {
        select {
        case <-lb.queue:
        default:
        }
    }
}

// Allow returns true if the request fits in the queue
func (lb *LeakyBucket) Allow() bool {
    select {
    case lb.queue <- struct{}{}:
        return true
    default:
        return false // queue full — request rejected
    }
}

Use leaky bucket when you’re calling a downstream service that cannot handle bursts at all — for example, a legacy system that processes exactly N requests per second and queues or crashes on more. The constant drain prevents any burst from being forwarded.

Sliding Window: Accurate Short-Term Limiting

Token bucket and leaky bucket operate on approximations — they don’t track exactly how many requests happened in the last N seconds. The sliding window algorithm does, at the cost of storing a timestamp per request:

type SlidingWindow struct {
    mu       sync.Mutex
    requests []time.Time
    maxReqs  int
    window   time.Duration
}

func NewSlidingWindow(maxReqs int, window time.Duration) *SlidingWindow {
    return &SlidingWindow{
        requests: make([]time.Time, 0, maxReqs),
        maxReqs:  maxReqs,
        window:   window,
    }
}

func (sw *SlidingWindow) Allow() bool {
    sw.mu.Lock()
    defer sw.mu.Unlock()

    now := time.Now()
    cutoff := now.Add(-sw.window)

    // Evict requests older than the window
    i := 0
    for i < len(sw.requests) && sw.requests[i].Before(cutoff) {
        i++
    }
    sw.requests = sw.requests[i:]

    if len(sw.requests) < sw.maxReqs {
        sw.requests = append(sw.requests, now)
        return true
    }
    return false
}

Sliding window is more accurate but uses more memory (one time.Time per request in the window) and is harder to share across processes. It’s best for single-instance services where precision matters — for example, limiting exactly 100 logins per 15-minute window.

Choosing the Right Algorithm

For most Go services, golang.org/x/time/rate (token bucket) is the right default. It handles the common case well, is well-tested, and has the context-aware Wait method that works naturally with Go’s cancellation model.

Communicating Rate Limits to Clients

Rate limiting is only useful if clients can respond to it. Follow standard HTTP conventions:

func rateLimitResponse(w http.ResponseWriter, retryAfterSecs int) {
    w.Header().Set("X-RateLimit-Limit", "100")
    w.Header().Set("X-RateLimit-Remaining", "0")
    w.Header().Set("X-RateLimit-Reset", strconv.FormatInt(time.Now().Add(time.Duration(retryAfterSecs)*time.Second).Unix(), 10))
    w.Header().Set("Retry-After", strconv.Itoa(retryAfterSecs))
    http.Error(w, `{"error":"rate limit exceeded"}`, http.StatusTooManyRequests)
}

Retry-After tells clients exactly when to try again, which reduces thundering herd problems where all rejected clients retry simultaneously.

Common Mistakes

Not propagating context through the limiter. If you use limiter.Wait(ctx) instead of limiter.Allow(), you get proper cancellation — the goroutine doesn’t wait forever if the request is cancelled upstream.

Using a global limiter per service without per-client limits. A single noisy client can exhaust the budget for everyone. Always pair a global limiter (protecting your service) with per-client limits (protecting fair access).

Forgetting the burst parameter. Setting rate.NewLimiter(10, 1) allows only 10 requests per second with no burst — even a brief pause followed by 2 simultaneous requests will reject one. Match burst size to realistic client behavior.

Applying rate limiting after expensive work. Rate limit middleware should run as early as possible in the request handler chain, before authentication, database lookups, or business logic.

Summary

• Use a buffered channel semaphore to cap concurrent operations — it’s idiomatic, cancellable, and requires no dependencies
• Reach for golang.org/x/time/rate for production rate limiting — it’s the standard, handles context correctly, and supports all three usage modes
• Token bucket allows bursts; leaky bucket enforces constant rate; sliding window gives exact counts
• Always implement per-client limits alongside global limits for fair multi-tenant APIs
• Set Retry-After headers so clients know when to try again without hammering your service

Resources

• golang.org/x/time/rate
• golang.org/x/sync/semaphore
• Token bucket algorithm
• Google Cloud: Rate limiting strategies
• Go Blog: Concurrency patterns