Microservices Architecture with Go

Introduction

Microservices architecture has become the de facto standard for building large-scale, distributed systems. Go’s simplicity, performance, and built-in concurrency features make it an excellent choice for implementing microservices. This guide covers the principles, patterns, and practices for building robust microservices with Go.

Microservices architecture breaks down monolithic applications into small, independent services that communicate over the network. Each service is responsible for a specific business capability and can be developed, deployed, and scaled independently.

Core Microservices Principles

Service Independence

Each microservice should be independently deployable and scalable. This means:

Separate codebases and repositories
Independent databases (database per service pattern)
Loose coupling between services
Clear service boundaries

Single Responsibility

Each microservice should have a single, well-defined responsibility. This principle ensures:

Easier maintenance and testing
Better code organization
Clearer service contracts
Reduced complexity

Resilience and Fault Tolerance

Microservices must handle failures gracefully:

Circuit breakers for failing services
Retry logic with exponential backoff
Timeout management
Graceful degradation

Good: Proper Microservice Design

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"time"

	"github.com/gorilla/mux"
)

// User represents a user in the system
type User struct {
	ID    string `json:"id"`
	Name  string `json:"name"`
	Email string `json:"email"`
}

// UserService handles user-related operations
type UserService struct {
	db map[string]User
}

// NewUserService creates a new user service
func NewUserService() *UserService {
	return &UserService{
		db: make(map[string]User),
	}
}

// GetUser retrieves a user by ID
func (s *UserService) GetUser(ctx context.Context, id string) (*User, error) {
	// Check context for cancellation
	select {
	case <-ctx.Done():
		return nil, ctx.Err()
	default:
	}

	user, exists := s.db[id]
	if !exists {
		return nil, fmt.Errorf("user not found: %s", id)
	}
	return &user, nil
}

// CreateUser creates a new user
func (s *UserService) CreateUser(ctx context.Context, user User) (*User, error) {
	select {
	case <-ctx.Done():
		return nil, ctx.Err()
	default:
	}

	s.db[user.ID] = user
	return &user, nil
}

// HTTP Handlers
func (s *UserService) handleGetUser(w http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	id := vars["id"]

	// Create context with timeout
	ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
	defer cancel()

	user, err := s.GetUser(ctx, id)
	if err != nil {
		http.Error(w, err.Error(), http.StatusNotFound)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(user)
}

func (s *UserService) handleCreateUser(w http.ResponseWriter, r *http.Request) {
	var user User
	if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}

	ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
	defer cancel()

	created, err := s.CreateUser(ctx, user)
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusCreated)
	json.NewEncoder(w).Encode(created)
}

// Service represents a microservice
type Service struct {
	name    string
	port    string
	router  *mux.Router
	service *UserService
}

// NewService creates a new service
func NewService(name, port string) *Service {
	return &Service{
		name:    name,
		port:    port,
		router:  mux.NewRouter(),
		service: NewUserService(),
	}
}

// RegisterRoutes registers HTTP routes
func (s *Service) RegisterRoutes() {
	s.router.HandleFunc("/users/{id}", s.service.handleGetUser).Methods("GET")
	s.router.HandleFunc("/users", s.service.handleCreateUser).Methods("POST")
	s.router.HandleFunc("/health", s.handleHealth).Methods("GET")
}

// handleHealth provides a health check endpoint
func (s *Service) handleHealth(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(map[string]string{
		"status": "healthy",
		"service": s.name,
	})
}

// Start starts the service
func (s *Service) Start() error {
	s.RegisterRoutes()
	log.Printf("Starting %s on port %s", s.name, s.port)
	return http.ListenAndServe(":"+s.port, s.router)
}

func main() {
	service := NewService("user-service", "8080")
	if err := service.Start(); err != nil {
		log.Fatal(err)
	}
}

This example demonstrates:

Clear service boundaries with dedicated handlers
Context-based cancellation and timeouts
Proper error handling
Health check endpoints
Structured service initialization

Bad: Monolithic Service Design

package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"time"
)

// BAD: Everything in one handler
func handleRequest(w http.ResponseWriter, r *http.Request) {
	// No context management
	// No timeout handling
	// Mixed concerns

	// User logic
	var user map[string]interface{}
	json.NewDecoder(r.Body).Decode(&user)

	// Order logic
	var order map[string]interface{}
	json.NewDecoder(r.Body).Decode(&order)

	// Payment logic
	var payment map[string]interface{}
	json.NewDecoder(r.Body).Decode(&payment)

	// All business logic mixed together
	// No separation of concerns
	// Difficult to test
	// Difficult to scale

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
}

// BAD: No health checks
func main() {
	http.HandleFunc("/", handleRequest)
	http.ListenAndServe(":8080", nil)
}

Problems with this approach:

No service isolation
Mixed business logic
No health monitoring
No context management
Difficult to scale independently

Service Communication Patterns

Synchronous Communication (REST/HTTP)

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"time"
)

// OrderService calls UserService
type OrderService struct {
	userServiceURL string
	client         *http.Client
}

// NewOrderService creates a new order service
func NewOrderService(userServiceURL string) *OrderService {
	return &OrderService{
		userServiceURL: userServiceURL,
		client: &http.Client{
			Timeout: 5 * time.Second,
		},
	}
}

// GetUserInfo retrieves user information from UserService
func (s *OrderService) GetUserInfo(ctx context.Context, userID string) (map[string]interface{}, error) {
	url := fmt.Sprintf("%s/users/%s", s.userServiceURL, userID)

	req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
	if err != nil {
		return nil, err
	}

	resp, err := s.client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("failed to get user info: %w", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		body, _ := io.ReadAll(resp.Body)
		return nil, fmt.Errorf("user service error: %s", string(body))
	}

	var user map[string]interface{}
	if err := json.NewDecoder(resp.Body).Decode(&user); err != nil {
		return nil, err
	}

	return user, nil
}

// CreateOrder creates a new order
func (s *OrderService) CreateOrder(ctx context.Context, userID string, items []string) (map[string]interface{}, error) {
	// Get user info first
	user, err := s.GetUserInfo(ctx, userID)
	if err != nil {
		return nil, err
	}

	order := map[string]interface{}{
		"user":  user,
		"items": items,
		"timestamp": time.Now(),
	}

	return order, nil
}

Asynchronous Communication (Message Queues)

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"time"
)

// Event represents a domain event
type Event struct {
	Type      string    `json:"type"`
	Data      interface{} `json:"data"`
	Timestamp time.Time `json:"timestamp"`
}

// EventBus defines the interface for event publishing
type EventBus interface {
	Publish(ctx context.Context, event Event) error
	Subscribe(ctx context.Context, eventType string, handler EventHandler) error
}

// EventHandler processes events
type EventHandler func(ctx context.Context, event Event) error

// SimpleEventBus is a basic in-memory event bus
type SimpleEventBus struct {
	subscribers map[string][]EventHandler
}

// NewSimpleEventBus creates a new event bus
func NewSimpleEventBus() *SimpleEventBus {
	return &SimpleEventBus{
		subscribers: make(map[string][]EventHandler),
	}
}

// Publish publishes an event
func (b *SimpleEventBus) Publish(ctx context.Context, event Event) error {
	handlers, exists := b.subscribers[event.Type]
	if !exists {
		return nil
	}

	for _, handler := range handlers {
		go func(h EventHandler) {
			if err := h(ctx, event); err != nil {
				log.Printf("Error handling event: %v", err)
			}
		}(handler)
	}

	return nil
}

// Subscribe subscribes to events
func (b *SimpleEventBus) Subscribe(ctx context.Context, eventType string, handler EventHandler) error {
	b.subscribers[eventType] = append(b.subscribers[eventType], handler)
	return nil
}

// OrderCreatedEvent represents an order creation event
type OrderCreatedEvent struct {
	OrderID string `json:"order_id"`
	UserID  string `json:"user_id"`
	Total   float64 `json:"total"`
}

// OrderService publishes events
type OrderServiceAsync struct {
	eventBus EventBus
}

// NewOrderServiceAsync creates a new async order service
func NewOrderServiceAsync(eventBus EventBus) *OrderServiceAsync {
	return &OrderServiceAsync{
		eventBus: eventBus,
	}
}

// CreateOrder creates an order and publishes an event
func (s *OrderServiceAsync) CreateOrder(ctx context.Context, userID string, total float64) error {
	orderID := fmt.Sprintf("order-%d", time.Now().Unix())

	event := Event{
		Type: "order.created",
		Data: OrderCreatedEvent{
			OrderID: orderID,
			UserID:  userID,
			Total:   total,
		},
		Timestamp: time.Now(),
	}

	return s.eventBus.Publish(ctx, event)
}

// NotificationService listens for order events
type NotificationService struct {
	eventBus EventBus
}

// NewNotificationService creates a new notification service
func NewNotificationService(eventBus EventBus) *NotificationService {
	return &NotificationService{
		eventBus: eventBus,
	}
}

// Start starts the notification service
func (s *NotificationService) Start(ctx context.Context) error {
	return s.eventBus.Subscribe(ctx, "order.created", s.handleOrderCreated)
}

// handleOrderCreated handles order creation events
func (s *NotificationService) handleOrderCreated(ctx context.Context, event Event) error {
	data, ok := event.Data.(map[string]interface{})
	if !ok {
		return fmt.Errorf("invalid event data")
	}

	log.Printf("Sending notification for order: %v", data)
	return nil
}

Service Discovery

package main

import (
	"context"
	"fmt"
	"sync"
	"time"
)

// ServiceInstance represents a service instance
type ServiceInstance struct {
	ID       string
	Name     string
	Host     string
	Port     int
	Metadata map[string]string
}

// ServiceRegistry manages service instances
type ServiceRegistry interface {
	Register(ctx context.Context, instance ServiceInstance) error
	Deregister(ctx context.Context, instanceID string) error
	GetInstances(ctx context.Context, serviceName string) ([]ServiceInstance, error)
	Watch(ctx context.Context, serviceName string) (<-chan ServiceInstance, error)
}

// SimpleServiceRegistry is a basic in-memory registry
type SimpleServiceRegistry struct {
	mu        sync.RWMutex
	instances map[string]ServiceInstance
	watchers  map[string][]chan ServiceInstance
}

// NewSimpleServiceRegistry creates a new registry
func NewSimpleServiceRegistry() *SimpleServiceRegistry {
	return &SimpleServiceRegistry{
		instances: make(map[string]ServiceInstance),
		watchers:  make(map[string][]chan ServiceInstance),
	}
}

// Register registers a service instance
func (r *SimpleServiceRegistry) Register(ctx context.Context, instance ServiceInstance) error {
	r.mu.Lock()
	defer r.mu.Unlock()

	r.instances[instance.ID] = instance

	// Notify watchers
	if watchers, exists := r.watchers[instance.Name]; exists {
		for _, watcher := range watchers {
			select {
			case watcher <- instance:
			case <-ctx.Done():
				return ctx.Err()
			}
		}
	}

	return nil
}

// Deregister deregisters a service instance
func (r *SimpleServiceRegistry) Deregister(ctx context.Context, instanceID string) error {
	r.mu.Lock()
	defer r.mu.Unlock()

	delete(r.instances, instanceID)
	return nil
}

// GetInstances gets all instances of a service
func (r *SimpleServiceRegistry) GetInstances(ctx context.Context, serviceName string) ([]ServiceInstance, error) {
	r.mu.RLock()
	defer r.mu.RUnlock()

	var instances []ServiceInstance
	for _, instance := range r.instances {
		if instance.Name == serviceName {
			instances = append(instances, instance)
		}
	}

	return instances, nil
}

// Watch watches for service changes
func (r *SimpleServiceRegistry) Watch(ctx context.Context, serviceName string) (<-chan ServiceInstance, error) {
	r.mu.Lock()
	defer r.mu.Unlock()

	ch := make(chan ServiceInstance, 10)
	r.watchers[serviceName] = append(r.watchers[serviceName], ch)

	return ch, nil
}

// LoadBalancer distributes requests across service instances
type LoadBalancer struct {
	registry ServiceRegistry
	index    int
	mu       sync.Mutex
}

// NewLoadBalancer creates a new load balancer
func NewLoadBalancer(registry ServiceRegistry) *LoadBalancer {
	return &LoadBalancer{
		registry: registry,
		index:    0,
	}
}

// GetNextInstance returns the next service instance (round-robin)
func (lb *LoadBalancer) GetNextInstance(ctx context.Context, serviceName string) (*ServiceInstance, error) {
	instances, err := lb.registry.GetInstances(ctx, serviceName)
	if err != nil {
		return nil, err
	}

	if len(instances) == 0 {
		return nil, fmt.Errorf("no instances available for service: %s", serviceName)
	}

	lb.mu.Lock()
	defer lb.mu.Unlock()

	instance := instances[lb.index%len(instances)]
	lb.index++

	return &instance, nil
}

Best Practices

1. API Versioning

Always version your APIs to maintain backward compatibility:

// Good: Versioned endpoints
router.HandleFunc("/v1/users/{id}", handleGetUserV1).Methods("GET")
router.HandleFunc("/v2/users/{id}", handleGetUserV2).Methods("GET")

2. Logging and Tracing

Implement structured logging for debugging:

type Logger struct {
	serviceName string
}

func (l *Logger) Log(level, message string, fields map[string]interface{}) {
	fields["service"] = l.serviceName
	fields["timestamp"] = time.Now()
	// Log to centralized logging system
}

3. Configuration Management

Externalize configuration for different environments:

type Config struct {
	ServiceName string
	Port        string
	Environment string
	LogLevel    string
}

func LoadConfig() *Config {
	return &Config{
		ServiceName: os.Getenv("SERVICE_NAME"),
		Port:        os.Getenv("PORT"),
		Environment: os.Getenv("ENVIRONMENT"),
		LogLevel:    os.Getenv("LOG_LEVEL"),
	}
}

4. Graceful Shutdown

Handle shutdown signals properly:

func (s *Service) GracefulShutdown(ctx context.Context) error {
	done := make(chan error, 1)
	go func() {
		done <- s.server.Shutdown(ctx)
	}()

	select {
	case err := <-done:
		return err
	case <-ctx.Done():
		return ctx.Err()
	}
}

Common Pitfalls

1. Tight Coupling

Avoid direct dependencies between services. Use event-driven or API-based communication.

2. Shared Databases

Each service should have its own database. Sharing databases creates tight coupling.

3. Synchronous Chains

Avoid long chains of synchronous calls. Use asynchronous communication when possible.

4. Insufficient Monitoring

Always implement comprehensive monitoring and logging for distributed systems.

Resources

Summary

Microservices architecture with Go provides a powerful foundation for building scalable, distributed systems. Key takeaways:

Design services with clear boundaries and single responsibilities
Choose appropriate communication patterns (synchronous vs asynchronous)
Implement proper service discovery and load balancing
Monitor and log comprehensively
Handle failures gracefully with resilience patterns
Version APIs and manage configuration externally
Implement graceful shutdown procedures

By following these principles and patterns, you can build robust microservices that scale effectively and maintain high availability.