Skip to main content
โšก Calmops

Software Testing Strategies: Beyond Unit Tests

Introduction

Unit tests form the foundation of software testing, but comprehensive quality requires much more. Modern applications need multiple testing layers, from fast unit tests to slow but thorough end-to-end tests. This guide explores testing strategies beyond unit testing that help build reliable software.

The Testing Pyramid

Understanding the Pyramid

        /\
       /E2E\          Few, Slow, Expensive
      /----\
     /Integration\    Medium frequency
    /------ ----\
   /   Unit Tests  \   Many, Fast, Cheap
  /----------------\

Balancing Tests

  • Many unit tests: Fast feedback, catch bugs early
  • Fewer integration tests: Test component interactions
  • Minimal E2E tests: Critical user journeys only

Integration Testing

What It Tests

Integration tests verify that components work together:

  • Database interactions
  • API integrations
  • Service communication
  • External dependencies

Database Testing

import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

@pytest.fixture
def test_db():
    engine = create_engine("sqlite:///:memory:")
    Base.metadata.create_all(engine)
    yield sessionmaker(bind=engine)()
    Base.metadata.drop_all(engine)

def test_user_creation(test_db):
    user = User(name="John", email="[email protected]")
    test_db.add(user)
    test_db.commit()
    
    retrieved = test_db.query(User).first()
    assert retrieved.name == "John"

API Testing

import pytest
from fastapi.testclient import TestClient

@pytest.fixture
def client():
    from main import app
    return TestClient(app)

def test_create_order(client, test_db):
    response = client.post("/orders", json={
        "items": [{"product_id": 1, "quantity": 2}],
        "customer_id": 1
    })
    
    assert response.status_code == 201
    data = response.json()
    assert data["id"] is not None
    assert len(data["items"]) == 1

Service Integration

def test_order_notification_integration():
    # Set up mock notification service
    with mock.patch('notifications.send') as mock_send:
        order_service = OrderService(notification_service)
        
        order = order_service.create_order(VALID_ORDER)
        
        mock_send.assert_called_once()
        call_args = mock_send.call_args
        assert "order_id" in call_args[0][0]

End-to-End Testing

When to Use E2E

E2E tests verify complete user flows:

  • Critical user journeys
  • Payment flows
  • Authentication flows
  • Core business processes

Playwright Example

from playwright.sync_api import sync_playwright

def test_user_checkout():
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page()
        
        # Navigate to store
        page.goto("https://store.example.com")
        
        # Browse products
        page.click(".product:first-child")
        
        # Add to cart
        page.click("#add-to-cart")
        
        # Checkout
        page.click("#checkout")
        page.fill("#email", "[email protected]")
        page.fill("#card", "4242424242424242")
        page.click("#pay")
        
        # Verify success
        assert "Thank you" in page.content()
        
        browser.close()

Best Practices

  • Test critical paths only
  • Run in parallel when possible
  • Use realistic data
  • Clean up after tests

Property-Based Testing

Concept

Instead of testing specific inputs, test properties that should hold for all inputs:

from hypothesis import given, strategies as st

@given(st.lists(st.integers(min_value=1, max_value=100)))
def test_sorting_properties(nums):
    sorted_nums = sorted(nums)
    
    # Property 1: Same length
    assert len(sorted_nums) == len(nums)
    
    # Property 2: Sorted order
    assert all(sorted_nums[i] <= sorted_nums[i+1] 
               for i in range(len(sorted_nums)-1))
    
    # Property 3: Same elements
    assert sorted_nums == sorted(nums)

Use Cases

  • Cryptographic operations
  • Data transformations
  • Serialization/deserialization
  • Algorithm verification

Python Libraries

  • Hypothesis: Most popular property testing
  • FAST: Fuzzing for property testing
  • AutoPyTest: AI-assisted property discovery

Contract Testing

Purpose

Ensure services can communicate:

  • Frontend-backend compatibility
  • Microservice interfaces
  • API versions
  • Provider-consumer agreements

Pact Example

import pytest
from pact import Consumer, Provider

@pytest.fixture
def pact():
    return Consumer('web-app').has_pact_with(Provider('user-service'))

def test_get_user(pact):
    pact.given('user exists').upon_receiving(
        'a request for user'
    ).with_request(
        method='GET',
        path='/users/1'
    ).will_respond_with(
        status=200,
        body={
            'id': 1,
            'name': 'John'
        }
    )
    
    with pact:
        response = requests.get('http://localhost:8000/users/1')
        assert response.json()['name'] == 'John'

Chaos Engineering

Purpose

Test system resilience by deliberately introducing failures:

  • Network failures
  • Service downtime
  • Resource exhaustion
  • Latency injection

Tools

  • Chaos Monkey: Netflix’s chaos tool
  • Litmus: Kubernetes chaos
  • Gremlin: Commercial chaos platform

Example

from chaos import experiment

@experiment
def test_payment_timeout():
    # Inject 5s delay into payment service
    with chaos.inject().delay("payment-service", seconds=5):
        response = client.post("/checkout")
        
        # System should handle gracefully
        assert response.status_code in [200, 503]
        if response.status_code == 200:
            # Should handle with timeout
            assert response.elapsed < 10

AI-Assisted Testing

Current Capabilities

AI helps with:

  • Test generation from code
  • Test maintenance
  • Bug detection
  • Flaky test identification

Tools

  • Diffblue: Unit test generation
  • Kite: Python test assistance
  • Copilot: Code and test suggestions

Human + AI

Best results combine AI efficiency with human judgment:

  • AI generates initial tests
  • Humans review and refine
  • Humans add edge cases
  • Humans verify critical assertions

Testing Strategy Framework

By Application Type

Web Applications

  • Unit: Business logic
  • Integration: API endpoints
  • E2E: Checkout, auth, search

APIs

  • Contract: All endpoints
  • Integration: Database, external services
  • Performance: Load testing

Data Pipelines

  • Unit: Transform functions
  • Integration: Pipeline steps
  • Data quality: Schema, completeness

By Risk Level

Risk Testing Level Investment
High Full coverage High
Medium Key paths Medium
Low Smoke tests Low

Measuring Test Quality

Metrics

  • Coverage: % of code tested
  • Mutation score: Are tests catching bugs?
  • Flakiness: Reliability over time
  • Speed: How fast do tests run?

Tools

  • Coverage.py: Python coverage
  • Mutmut: Mutation testing
  • TestRail: Test management

Conclusion

Comprehensive testing requires multiple layers working together. Unit tests provide fast feedback, integration tests verify component collaboration, and E2E tests ensure critical user journeys work. Property-based testing and chaos engineering add robustness. Build your testing strategy based on your application’s risk profile and user expectations.


Resources

Comments