Introduction
You can have 100% code coverage but still have bad tests. Mutation testing verifies test quality by introducing small changes (mutations) to your code and checking if tests catch them.
How Mutation Testing Works
┌─────────────────────────────────────────────────────────────┐
│ Mutation Testing Process │
├─────────────────────────────────────────────────────────────┤
│ │
│ Original Code: │
│ ┌─────────────────────────────────────────┐ │
│ │ function add(a, b) { │ │
│ │ return a + b; │ │
│ │ } │ │
│ └─────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ Mutation: Change + to - │
│ ┌─────────────────────────────────────────┐ │
│ │ function add(a, b) { │ │
│ │ return a - b; ← MUTANT │ │
│ │ } │ │
│ └─────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ Run Tests: │
│ ┌─────────────────────────────────────────┐ │
│ │ add(2, 2) should return 4 │ │
│ │ │ │
│ │ Mutant survives? ❌ (test failed) │ │
│ │ Tests caught it! ✓ │ │
│ └─────────────────────────────────────────┘ │
│ │
│ Mutation Score = Caught / Total Mutants │
│ │
└─────────────────────────────────────────────────────────────┘
JavaScript/TypeScript: Stryker
Setup
npm install -D @stryker-mutator/core
npx stryker init
Configuration
// stryker.conf.json
{
"$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
"mutator": "typescript",
"packageManager": "npm",
"reporters": ["html", "clear-text", "dashboard"],
"buildCommand": "npm run build",
"testRunner": "command",
"commandRunner": {
"command": "npm test"
},
"mutate": [
"src/**/*.ts",
"!src/**/*.spec.ts"
],
"thresholds": {
"high": 80,
"low": 70,
"break": 60
}
}
Running
# Run mutation testing
npx stryker run
# Output:
# Mutant killed: 45/50 (90%)
# ┌───────────────────────────────────────┐
# │ File │ Mutation score │
# │ ──────────────│───────────────────── │
# │ math.ts │ 95% │
# │ string.ts │ 85% │
# │ utils.ts │ 75% ⚠️ │
# └───────────────────────────────────────┘
Java: PIT
Setup
<!-- pom.xml -->
<build>
<plugins>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.15.0</version>
</plugin>
</plugins>
</build>
Running
mvn org.pitest:pitest-maven:mutationCoverage
Results
================================================================================
>> mutation coverage report
================================================================================
Classes : 85% (34/40)
Methods : 80% (120/150)
Mutations: 75% (150/200)
UNCOVERED MUTATIONS:
================================================================================
com.example.Utils.java:
Line 45: replaced + with - SURVIVED
Line 67: removed conditional - omitted < SURVIVED
What Gets Mutated
mutations:
- "Arithmetic: + → -, * → /, etc."
- "Boolean: true → false, && → ||"
- "Comparison: == → !=, < → >="
- "Conditional: removed if/else branches"
- "String: '' → '', .length → 0"
- "Array: [0] → arr.first()"
Interpretation
# Mutation score interpretation
scores:
above_80:
label: "Excellent"
description: "Tests are catching most bugs"
70_80:
label: "Good"
description: "Room for improvement"
60_70:
label: "Warning"
description: "Some bugs may slip through"
below_60:
label: "Critical"
description: "Tests not effective"
Key Takeaways
- Mutation testing - Verifies test quality, not just coverage
- Stryker - JavaScript/TypeScript
- PIT - Java
- Goal - Catch as many mutants as possible
Comments