Introduction
Traditional testing uses specific examples. Property-based testing throws thousands of random inputs at your code, finding bugs you’d never think to test. This guide covers property-based testing with fast-check and Hypothesis.
Traditional vs Property-Based
┌─────────────────────────────────────────────────────────────┐
│ Traditional vs Property-Based Testing │
├─────────────────────────────────────────────────────────────┤
│ │
│ TRADITIONAL (Example-Based): │
│ │
│ test('sort works', () => { │
│ expect(sort([3, 1, 2])).toEqual([1, 2, 3]); │
│ }); │
│ │
│ ✓ Tests specific cases │
│ ✓ Easy to understand │
│ ✗ May miss edge cases │
│ │
│ PROPERTY-BASED: │
│ │
│ test('sort is correct', () => { │
│ fc.assert(fc.property(fc.array(fc.integer()), arr => { │
│ const sorted = sort(arr); │
│ return isSorted(sorted) && sameElements(arr, sorted);│
│ })); │
│ }); │
│ │
│ ✓ Tests thousands of random cases │
│ ✓ Finds edge cases automatically │
│ ✓ More confidence in correctness │
│ │
└─────────────────────────────────────────────────────────────┘
Key Concepts
Properties
A property is a statement that’s always true:
properties:
- "Sorting a list produces a sorted list"
- "Reversing twice returns original"
- "Adding two numbers equals adding in any order"
- "JSON.parse(JSON.stringify(x)) equals x"
fast-check (JavaScript/TypeScript)
Setup
npm install fast-check
Basic Example
import { fc, test } from 'fast-check';
// Property: Sorting produces sorted output
test('sort produces sorted array', () => {
fc.assert(
fc.property(fc.array(fc.integer()), (arr) => {
const sorted = [...arr].sort((a, b) => a - b);
// Check sorted
for (let i = 1; i < sorted.length; i++) {
if (sorted[i] < sorted[i - 1]) {
return false;
}
}
return true;
})
);
});
// Property: Reversing twice returns original
test('reverse twice returns original', () => {
fc.assert(
fc.property(fc.array(fc.integer()), (arr) => {
const reversed = [...arr].reverse();
const doubleReversed = [...reversed].reverse();
return JSON.stringify(arr) === JSON.stringify(doubleReversed);
})
);
});
Custom Arbitraries
// Generate valid email addresses
const emailArb = fc.record({
username: fc.string({ minLength: 1 }),
domain: fc.oneof(
fc.constant('gmail.com'),
fc.constant('outlook.com'),
fc.constant('company.com')
),
}).map(({ username, domain }) => `${username}@${domain}`);
fc.assert(
fc.property(emailArb, (email) => {
return isValidEmail(email); // Your validation function
})
);
// Generate valid JSON
const jsonValueArb: fc.Arbitrary<unknown> = fc.oneof(
fc.constant(null),
fc.boolean(),
fc.integer(),
fc.string(),
fc.array(jsonValueArb),
fc.dictionary(jsonValueArb)
);
Filtering and Shrinking
// Generate non-empty arrays
test('sort non-empty arrays', () => {
fc.assert(
fc.property(
fc.array(fc.integer(), { minLength: 1 }),
(arr) => {
return sort(arr).length > 0;
}
)
);
});
// Filter invalid inputs
test('parse valid JSON', () => {
fc.assert(
fc.property(
fc.string().filter(s => {
try { JSON.parse(s); return true; }
catch { return false; }
}),
(str) => {
const parsed = JSON.parse(str);
return typeof parsed === 'object';
}
)
);
});
Testing API Functions
Database Operations
import { fc } from 'fast-check';
// Property: Creating user with valid data succeeds
test('createUser accepts valid input', () => {
fc.assert(
fc.property(
fc.record({
name: fc.string({ minLength: 1, maxLength: 100 }),
email: fc.emailAddress(),
age: fc.integer({ min: 0, max: 150 }),
}),
async (userData) => {
const user = await createUser(userData);
return user.name === userData.name &&
user.email === userData.email;
}
)
);
});
Business Logic
// Property: Discount never exceeds original price
test('discount is valid', () => {
fc.assert(
fc.property(
fc.float({ min: 0.01, max: 10000 }),
fc.float({ min: 0, max: 100 }),
(price, discount) => {
const finalPrice = calculateDiscount(price, discount);
return finalPrice >= 0 && finalPrice <= price;
}
)
);
});
Hypothesis (Python)
Setup
pip install hypothesis
Basic Example
from hypothesis import given, settings, Verbosity
import hypothesis.strategies as st
@given(st.lists(st.integers()))
def test_sort_returns_sorted(arr):
sorted_arr = sorted(arr)
# Check each element is in order
for i in range(len(sorted_arr) - 1):
assert sorted_arr[i] <= sorted_arr[i + 1]
@given(st.lists(st.integers(), min_size=1))
def test_sort_preserves_length(arr):
sorted_arr = sorted(arr)
assert len(sorted_arr) == len(arr)
Advanced Strategies
from hypothesis import given, st
import re
# Custom strategies
@given(st.from_regex(r"[a-z]+@[a-z]+\.(com|org|net)"))
def test_valid_email_format(email):
assert re.match(r"[a-z]+@[a-z]+\.(com|org|net)", email)
# Dependent generation
@given(
data=st.data()
)
def test_list_operations(data):
# Generate list size
size = data.draw(st.integers(min_value=1, max_value=100))
arr = data.draw(st.lists(st.integers(), min_size=size, max_size=size))
# Test property
assert len(sorted(arr)) == size
Key Takeaways
- Property-based testing - Tests thousands of random cases
- Finds edge cases - Bugs you’d never manually test
- Great for - Pure functions, data transformations, API validation
- Not great for - UI interactions, complex integrations
Comments