Monitoring: Prometheus and Grafana
Introduction
Monitoring is essential for production systems. This guide covers collecting metrics with Prometheus and visualizing with Grafana.
Good: Prometheus Metrics
Instrumenting Go Code
package main
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/gin-gonic/gin"
)
// โ
GOOD: Define metrics
var (
requestCount = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "http_requests_total",
Help: "Total HTTP requests",
},
[]string{"method", "path", "status"},
)
requestDuration = prometheus.NewHistogramVec(
prometheus.HistogramOpts{
Name: "http_request_duration_seconds",
Help: "HTTP request duration",
},
[]string{"method", "path"},
)
activeConnections = prometheus.NewGauge(
prometheus.GaugeOpts{
Name: "active_connections",
Help: "Active connections",
},
)
)
func init() {
prometheus.MustRegister(requestCount)
prometheus.MustRegister(requestDuration)
prometheus.MustRegister(activeConnections)
}
// โ
GOOD: Metrics middleware
func metricsMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now()
activeConnections.Inc()
defer activeConnections.Dec()
c.Next()
duration := time.Since(start).Seconds()
requestCount.WithLabelValues(
c.Request.Method,
c.Request.URL.Path,
strconv.Itoa(c.Writer.Status()),
).Inc()
requestDuration.WithLabelValues(
c.Request.Method,
c.Request.URL.Path,
).Observe(duration)
}
}
// โ
GOOD: Expose metrics
func setupMetrics(router *gin.Engine) {
router.GET("/metrics", gin.WrapH(promhttp.Handler()))
}
Good: Prometheus Configuration
prometheus.yml
# โ
GOOD: Prometheus configuration
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'go-app'
static_configs:
- targets: ['localhost:8080']
metrics_path: '/metrics'
Good: Grafana Dashboards
Docker Compose
# โ
GOOD: Prometheus and Grafana setup
version: '3.8'
services:
prometheus:
image: prom/prometheus:latest
ports:
- "9090:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
command:
- '--config.file=/etc/prometheus/prometheus.yml'
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- grafana_data:/var/lib/grafana
volumes:
grafana_data:
Best Practices
1. Use Appropriate Metric Types
// โ
GOOD: Counter for totals
requestCount.Inc()
// โ
GOOD: Gauge for current values
activeConnections.Set(float64(count))
// โ
GOOD: Histogram for distributions
requestDuration.Observe(duration)
2. Add Labels
// โ
GOOD: Meaningful labels
requestCount.WithLabelValues(method, path, status).Inc()
// โ BAD: No labels
requestCount.Inc()
3. Monitor Key Metrics
// โ
GOOD: Monitor important metrics
- Request count
- Request duration
- Error rate
- Active connections
- Database connections
Resources
- Prometheus: https://prometheus.io/
- Grafana: https://grafana.com/
- Go Prometheus Client: https://github.com/prometheus/client_golang
Summary
Prometheus and Grafana provide powerful monitoring capabilities. Instrument your code with metrics, configure Prometheus to scrape them, and visualize with Grafana dashboards. Proper monitoring enables proactive issue detection and performance optimization.
Comments