Analytics and Reporting in Go

Analytics in Go ranges from simple in-memory counters to streaming aggregations over millions of events. This guide covers the practical patterns: collecting and aggregating metrics, computing statistical summaries (percentiles, trends), grouping by dimensions, and generating reports in multiple formats.

For production metrics and dashboards, Prometheus is the standard choice. The patterns here are for embedded analytics — building reporting into your application rather than shipping to an external system.

Collecting Metrics

A metric is a named value with a timestamp and optional tags (dimensions that let you segment and filter):

type Metric struct {
    Name      string
    Value     float64
    Timestamp time.Time
    Tags      map[string]string  // e.g., {"region": "us-east", "status": "200"}
}

type MetricStore struct {
    mu      sync.Mutex
    metrics []Metric
}

func (ms *MetricStore) Record(name string, value float64, tags map[string]string) {
    ms.mu.Lock()
    ms.metrics = append(ms.metrics, Metric{
        Name:      name,
        Value:     value,
        Timestamp: time.Now(),
        Tags:      tags,
    })
    ms.mu.Unlock()
}

func (ms *MetricStore) Snapshot() []Metric {
    ms.mu.Lock()
    out := make([]Metric, len(ms.metrics))
    copy(out, ms.metrics)
    ms.mu.Unlock()
    return out
}

In production services, the MetricStore should cap its size and evict old metrics — an unbounded slice will exhaust memory over time. A common pattern is to aggregate on write rather than storing raw values.

Statistical Aggregations

Given a slice of values, compute the standard summary statistics:

type Stats struct {
    Count  int
    Sum    float64
    Mean   float64
    Min    float64
    Max    float64
    P50    float64  // median
    P95    float64
    P99    float64
}

func Aggregate(values []float64) Stats {
    if len(values) == 0 {
        return Stats{}
    }

    sorted := make([]float64, len(values))
    copy(sorted, values)
    sort.Float64s(sorted)

    n := len(sorted)
    var sum float64
    for _, v := range sorted {
        sum += v
    }

    return Stats{
        Count: n,
        Sum:   sum,
        Mean:  sum / float64(n),
        Min:   sorted[0],
        Max:   sorted[n-1],
        P50:   percentile(sorted, 50),
        P95:   percentile(sorted, 95),
        P99:   percentile(sorted, 99),
    }
}

func percentile(sorted []float64, p float64) float64 {
    if len(sorted) == 0 {
        return 0
    }
    // Nearest-rank method
    idx := int(math.Ceil(p/100*float64(len(sorted)))) - 1
    if idx < 0 { idx = 0 }
    if idx >= len(sorted) { idx = len(sorted) - 1 }
    return sorted[idx]
}

P95 and P99 latencies are more useful than mean for understanding tail latency — they tell you what 5% or 1% of your worst requests experience, which is usually what causes user complaints.

Grouping by Dimension

GroupBy partitions metrics by a tag value and returns a map of group → metric slice:

func GroupBy(metrics []Metric, tag string) map[string][]Metric {
    groups := make(map[string][]Metric)
    for _, m := range metrics {
        key := m.Tags[tag]
        if key == "" {
            key = "(unset)"
        }
        groups[key] = append(groups[key], m)
    }
    return groups
}

// Usage: analyze response times by HTTP status code
groups := GroupBy(store.Snapshot(), "status")
for status, metrics := range groups {
    values := make([]float64, len(metrics))
    for i, m := range metrics {
        values[i] = m.Value
    }
    stats := Aggregate(values)
    fmt.Printf("HTTP %s: count=%d, p95=%.1fms, p99=%.1fms\n",
        status, stats.Count, stats.P95*1000, stats.P99*1000)
}

Multiple dimensions: apply GroupBy sequentially, or extend to partition by a composite key like region/status.

Time-Bucketing

Time bucketing groups events into fixed-duration windows — the foundation of per-minute, per-hour, or per-day charts:

// Bucket returns a map of window-start-time → metrics in that window
func Bucket(metrics []Metric, window time.Duration) map[time.Time][]Metric {
    buckets := make(map[time.Time][]Metric)
    for _, m := range metrics {
        // Truncate timestamp to window boundary
        start := m.Timestamp.Truncate(window)
        buckets[start] = append(buckets[start], m)
    }
    return buckets
}

// Usage: request count per minute
buckets := Bucket(store.Snapshot(), time.Minute)

// Sort bucket keys for a time-ordered report
keys := make([]time.Time, 0, len(buckets))
for k := range buckets {
    keys = append(keys, k)
}
sort.Slice(keys, func(i, j int) bool { return keys[i].Before(keys[j]) })

for _, t := range keys {
    stats := Aggregate(valuesFrom(buckets[t]))
    fmt.Printf("%s: requests=%d, p95=%.1fms\n",
        t.Format("15:04"), stats.Count, stats.P95*1000)
}

Trend Analysis

A simple linear regression tells you whether a metric is increasing or decreasing over time:

// TrendSlope returns the slope of the best-fit line through the values.
// Positive slope = increasing trend, negative = decreasing.
func TrendSlope(values []float64) float64 {
    n := float64(len(values))
    if n < 2 {
        return 0
    }
    var sumX, sumY, sumXY, sumX2 float64
    for i, y := range values {
        x := float64(i)
        sumX += x
        sumY += y
        sumXY += x * y
        sumX2 += x * x
    }
    denominator := n*sumX2 - sumX*sumX
    if denominator == 0 {
        return 0
    }
    return (n*sumXY - sumX*sumY) / denominator
}

// If slope > 0 and error rate is the metric, investigate immediately
slope := TrendSlope(errorRates)
if slope > threshold {
    alertOps("error rate increasing: slope=%.4f", slope)
}

Report Generation

Reports summarize analytics for human consumption. Using an interface lets you generate text, JSON, or CSV without changing the aggregation logic:

type Report struct {
    Title     string
    Generated time.Time
    Sections  []Section
}

type Section struct {
    Name    string
    Stats   Stats
    Groups  map[string]Stats  // grouped stats, optional
}

type Formatter interface {
    Format(r Report) string
}

type TextFormatter struct{}

func (f TextFormatter) Format(r Report) string {
    var b strings.Builder
    fmt.Fprintf(&b, "=== %s ===\nGenerated: %s\n\n",
        r.Title, r.Generated.Format(time.RFC3339))
    for _, sec := range r.Sections {
        fmt.Fprintf(&b, "--- %s ---\n", sec.Name)
        fmt.Fprintf(&b, "  count: %d\n  mean:  %.2f\n  p95:   %.2f\n  p99:   %.2f\n\n",
            sec.Stats.Count, sec.Stats.Mean, sec.Stats.P95, sec.Stats.P99)
        if len(sec.Groups) > 0 {
            fmt.Fprintln(&b, "  By group:")
            for name, gs := range sec.Groups {
                fmt.Fprintf(&b, "    %-15s count=%-6d p95=%.2f\n", name, gs.Count, gs.P95)
            }
        }
    }
    return b.String()
}

type JSONFormatter struct{}

func (f JSONFormatter) Format(r Report) string {
    data, _ := json.MarshalIndent(r, "", "  ")
    return string(data)
}

type CSVFormatter struct{}

func (f CSVFormatter) Format(r Report) string {
    var b strings.Builder
    fmt.Fprintln(&b, "section,count,mean,p50,p95,p99")
    for _, sec := range r.Sections {
        fmt.Fprintf(&b, "%s,%d,%.2f,%.2f,%.2f,%.2f\n",
            sec.Name, sec.Stats.Count,
            sec.Stats.Mean, sec.Stats.P50, sec.Stats.P95, sec.Stats.P99)
    }
    return b.String()
}

Prometheus for Production

For production services, use Prometheus instead of rolling your own analytics. Prometheus provides a time-series database, query language (PromQL), and Grafana dashboards:

import "github.com/prometheus/client_golang/prometheus"

var (
    httpRequestDuration = prometheus.NewHistogramVec(
        prometheus.HistogramOpts{
            Name:    "http_request_duration_seconds",
            Help:    "HTTP request latency",
            Buckets: []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5},
        },
        []string{"method", "path", "status"},
    )
    httpRequestTotal = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total HTTP requests",
        },
        []string{"method", "path", "status"},
    )
)

func init() {
    prometheus.MustRegister(httpRequestDuration, httpRequestTotal)
}

// Middleware: record latency and count per route
func metricsMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        rw := &statusWriter{ResponseWriter: w, code: 200}
        next.ServeHTTP(rw, r)
        duration := time.Since(start).Seconds()
        status := strconv.Itoa(rw.code)
        httpRequestDuration.WithLabelValues(r.Method, r.URL.Path, status).Observe(duration)
        httpRequestTotal.WithLabelValues(r.Method, r.URL.Path, status).Inc()
    })
}

Expose the metrics endpoint:

import "github.com/prometheus/client_golang/prometheus/promhttp"
mux.Handle("/metrics", promhttp.Handler())

Then query in Grafana:

histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) — P95 latency
rate(http_requests_total[5m]) — requests per second
sum by (status) (rate(http_requests_total[5m])) — requests per second by status code

Summary

Aggregate with sort + percentile functions for P50/P95/P99 — more meaningful than mean for latency data
Group by tag dimension to understand which segment is causing problems (region, status code, user tier)
Time bucket with Timestamp.Truncate(window) for time-series charts
Linear regression slope detects whether a metric is trending up or down
Use a Formatter interface to generate the same report in text, JSON, and CSV without duplicating logic
For production services, Prometheus + Grafana covers analytics far better than custom code

Analytics and Reporting in Go

Collecting Metrics

Statistical Aggregations

Grouping by Dimension

Time-Bucketing

Trend Analysis

Report Generation

Prometheus for Production

Summary

Resources

Comments

Share this article

👍 Was this article helpful?