Introduction
A solid testing strategy is essential for delivering quality software. Without proper testing, bugs make it to production, causing issues for users and requiring expensive hotfixes. This comprehensive guide covers the testing pyramid and building effective test suites.
The testing pyramid provides a framework for balancing different types of tests. Understanding when to use each level helps you build comprehensive coverage while maintaining reasonable development velocity.
Whether you’re building a small application or a large distributed system, having the right test strategy ensures you can confidently ship changes without breaking existing functionality.
The Testing Pyramid
Understanding the Pyramid
The testing pyramid organizes tests by scope and speed:
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Testing Pyramid โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ โ
โ โโโโโโโโโโโโโโโ โ
โ โ E2E โ โ
โ โ (5-10%) โ โ
โ โ End-to-End โ โ
โ โ Full App โ โ
โ โโโโโโโโโโโโโโโ โ
โ โ
โ โโโโโโโโโโโโโโโโโโโโ โ
โ โ Integration โ โ
โ โ (20-30%) โ โ
โ โ Multiple โ โ
โ โ Components โ โ
โ โโโโโโโโโโโโโโโโโโโโ โ
โ โ
โ โโโโโโโโโโโโโโโโโโโโโโ โ
โ โ Unit โ โ
โ โ (60-70%) โ โ
โ โ Individual โ โ
โ โ Functions โ โ
โ โโโโโโโโโโโโโโโโโโโโโโ โ
โ โ
โ Speed: Fast โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Slow โ
โ Cost: Low โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ High โ
โ Coverage: High โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Low โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Why the Pyramid Matters
| Level | Count | Speed | Cost | Purpose |
|---|---|---|---|---|
| Unit | Many | Fast | Low | Individual functions |
| Integration | Medium | Medium | Medium | Component interaction |
| E2E | Few | Slow | High | Full user journeys |
Unit Testing
Writing Effective Unit Tests
import unittest
from dataclasses import dataclass
from typing import Optional
@dataclass
class User:
id: int
name: str
email: str
is_active: bool = True
class UserValidator:
"""Validates user data."""
@staticmethod
def validate(user: User) -> tuple[bool, Optional[str]]:
"""Validate user and return (is_valid, error_message)."""
if not user.name or len(user.name.strip()) == 0:
return False, "Name is required"
if len(user.name) > 100:
return False, "Name must be less than 100 characters"
if not user.email or '@' not in user.email:
return False, "Valid email is required"
return True, None
class TestUserValidator(unittest.TestCase):
"""Test cases for UserValidator."""
def test_valid_user(self):
"""Test that valid user passes validation."""
user = User(id=1, name="John", email="[email protected]")
is_valid, error = UserValidator.validate(user)
self.assertTrue(is_valid)
self.assertIsNone(error)
def test_empty_name_fails(self):
"""Test that empty name fails validation."""
user = User(id=1, name="", email="[email protected]")
is_valid, error = UserValidator.validate(user)
self.assertFalse(is_valid)
self.assertEqual(error, "Name is required")
def test_missing_email_fails(self):
"""Test that missing email fails."""
user = User(id=1, name="John", email="")
is_valid, error = UserValidator.validate(user)
self.assertFalse(is_valid)
self.assertEqual(error, "Valid email is required")
def test_invalid_email_format_fails(self):
"""Test that invalid email format fails."""
user = User(id=1, name="John", email="not-an-email")
is_valid, error = UserValidator.validate(user)
self.assertFalse(is_valid)
self.assertEqual(error, "Valid email is required")
def test_long_name_fails(self):
"""Test that too long name fails."""
user = User(id=1, name="A" * 101, email="[email protected]")
is_valid, error = UserValidator.validate(user)
self.assertFalse(is_valid)
self.assertEqual(error, "Name must be less than 100 characters")
# Test fixtures with setUp
class TestOrderProcessing(unittest.TestCase):
"""Test order processing logic."""
def setUp(self):
"""Set up test fixtures."""
self.order_data = {
'items': [
{'product_id': 1, 'quantity': 2, 'price': 10.00},
{'product_id': 2, 'quantity': 1, 'price': 25.00}
],
'customer_id': 123
}
def test_order_total_calculation(self):
"""Test order total is calculated correctly."""
total = calculate_order_total(self.order_data['items'])
self.assertEqual(total, 45.00) # 2*10 + 1*25
def test_empty_items_zero_total(self):
"""Test that empty items have zero total."""
self.order_data['items'] = []
total = calculate_order_total(self.order_data['items'])
self.assertEqual(total, 0)
# Helper function
def calculate_order_total(items: list) -> float:
"""Calculate total for order items."""
return sum(item['quantity'] * item['price'] for item in items)
if __name__ == '__main__':
unittest.main()
Using pytest
import pytest
# Simple test functions
def test_calculator_add():
"""Test basic addition."""
assert add(2, 3) == 5
def test_calculator_divide():
"""Test division."""
assert divide(10, 2) == 5
with pytest.raises(ZeroDivisionError):
divide(10, 0)
# Parametrized tests
@pytest.mark.parametrize("input,expected", [
(add(2, 3), 5),
(add(0, 0), 0),
(add(-1, 1), 0),
])
def test_add_parametrized(input, expected):
assert input == expected
# Fixtures
@pytest.fixture
def sample_user():
"""Fixture providing test user."""
return User(id=1, name="John", email="[email protected]")
def test_user_name(sample_user):
"""Test user name is correct."""
assert sample_user.name == "John"
# Mocking
from unittest.mock import Mock, patch
def test_api_call():
"""Test API call with mock."""
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {'data': 'test'}
with patch('requests.get', return_value=mock_response):
result = get_data()
assert result == {'data': 'test'}
Integration Testing
Testing Component Interactions
import pytest
from unittest.mock import Mock
class TestUserServiceIntegration:
"""Integration tests for UserService."""
@pytest.fixture
def mock_user_repo(self):
"""Mock user repository."""
return Mock()
@pytest.fixture
def mock_email_service(self):
"""Mock email service."""
return Mock()
@pytest.fixture
def user_service(self, mock_user_repo, mock_email_service):
"""Create UserService with mocked dependencies."""
return UserService(
user_repo=mock_user_repo,
email_service=mock_email_service
)
def test_create_user_sends_welcome_email(self, user_service, mock_email_service):
"""Test that creating user sends welcome email."""
mock_user_repo.create.return_value = User(
id=1, name="John", email="[email protected]"
)
user_service.create_user("John", "[email protected]")
mock_email_service.send_welcome.assert_called_once()
def test_create_user_persists_to_db(self, user_service, mock_user_repo):
"""Test that creating user saves to database."""
user_service.create_user("John", "[email protected]")
mock_user_repo.create.assert_called_once_with(
name="John", email="[email protected]"
)
class TestOrderWorkflow:
"""Test complete order workflow."""
def test_complete_order_flow(self, order_service, payment_service, inventory_service):
"""Test full order processing."""
# Setup mocks
inventory_service.check_available.return_value = True
payment_service.charge.return_value = True
# Execute
result = order_service.process_order(
customer_id=1,
items=[{'product_id': 1, 'quantity': 2}]
)
# Verify
assert result.status == 'completed'
inventory_service.reserve.assert_called_once()
payment_service.charge.assert_called_once()
End-to-End Testing
E2E with Playwright
import pytest
from playwright.sync_api import sync_playwright
@pytest.fixture
def browser_context():
"""Create browser context for testing."""
with sync_playwright() as p:
browser = p.chromium.launch()
context = browser.new_context()
yield context
context.close()
browser.close()
def test_user_login_flow(browser_context):
"""Test complete user login flow."""
page = browser_context.new_page()
# Navigate to login page
page.goto("https://example.com/login")
# Fill login form
page.fill('[data-testid="email"]', "[email protected]")
page.fill('[data-testid="password"]', "password123")
# Submit
page.click('[data-testid="login-button"]')
# Wait for redirect
page.wait_for_url("**/dashboard")
# Verify we're logged in
assert page.is_visible('[data-testid="user-menu"]')
page.close()
def test_shopping_cart_flow(browser_context):
"""Test shopping cart end-to-end."""
page = browser_context.new_page()
# Add item to cart
page.goto("/product/1")
page.click("button.add-to-cart")
# Go to cart
page.click("a.cart-link")
# Verify item in cart
assert page.is_visible(".cart-item")
assert "Product Name" in page.text_content(".cart-item")
# Checkout
page.click("button.checkout")
# Fill checkout form
page.fill("#name", "John Doe")
page.fill("#address", "123 Main St")
# Complete order
page.click("button.place-order")
# Verify success
assert page.is_visible(".order-confirmation")
page.close()
Test-Driven Development
TDD Cycle
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ TDD Cycle (Red-Green-Refactor) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ โ
โ 1. RED: Write failing test โ
โ โข Test describes desired behavior โ
โ โข Test fails because feature doesn't exist yet โ
โ โ
โ 2. GREEN: Make test pass โ
โ โข Write minimum code to pass โ
โ โข Don't over-engineer โ
โ โ
โ 3. REFACTOR: Improve code โ
โ โข Clean up implementation โ
โ โข Ensure tests still pass โ
โ โ
โ Repeat for each feature โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
TDD Example
# Step 1: RED - Write failing test
def test_calculator_power():
"""Test power function."""
assert power(2, 3) == 8 # Fails - function doesn't exist
# Step 2: GREEN - Make it pass
def power(base, exp):
"""Calculate base raised to exp power."""
return base ** exp # Passes!
# Step 3: REFACTOR - Clean up
def power(base: float, exp: int) -> float:
"""Calculate base raised to exp power.
Args:
base: The base number
exp: The exponent
Returns:
base raised to the power of exp
"""
return base ** exp
Test Coverage
Measuring and Improving Coverage
# Run with coverage
# pytest --cov=myapp --cov-report=html tests/
# .coveragerc
[run]
source = myapp
omit =
*/tests/*
*/migrations/*
[report]
exclude_lines =
pragma: no cover
if __name__ == .__main__.:
raise NotImplementedError
Best Practices
| Practice | Implementation |
|---|---|
| Test behavior | Test what, not how |
| AAA pattern | Arrange, Act, Assert |
| One assertion | Per test when possible |
| Descriptive names | test_user_cannot_login_with_wrong_password |
| Fast tests | Keep unit tests under 100ms |
| Independent tests | No dependencies between tests |
| Mock external | Don’t hit real APIs in unit tests |
Conclusion
A comprehensive testing strategy is essential for software quality. The testing pyramid provides a framework for balancing different test types, ensuring you have confidence in your code while maintaining reasonable development velocity.
Key takeaways:
- Start with unit tests - Most of your tests should be unit tests
- Add integration tests - Test component interactions
- Use E2E sparingly - Only for critical user journeys
- Follow AAA pattern - Arrange, Act, Assert
- Test behavior, not implementation - What, not how
- Keep tests fast - Under 100ms for unit tests
- Make tests independent - No shared state
By implementing these testing practices, you’ll build a robust test suite that gives you confidence to ship changes quickly.
Comments