ยท testing
Boost Your TypeScript Tests with Mutation Testing
Mutation testing evaluates the quality of your test suite by introducing small changes (mutations) to your code and checking if your tests can detect them. This tutorial will guide you through understanding mutation testing, setting up Stryker for TypeScript, and using it to enhance your test suite and code quality.
Contents
- What is Mutation Testing?
- Setting Up Stryker for TypeScript
- Code Mutation Examples
- Identifying Unnecessary Tests
- Finding Ineffective Source Code
- Disabling Mutations
- Testing Large Codebases
- Video Tutorial
What is Mutation Testing?
Mutation testing frameworks, like Stryker, create copies of your source code and introduce small changes to see if your test suite detects these changes. These changes are called mutants. If your tests are well-written, they will catch these mutants, resulting in what is known as "killing the mutant."
The Stryker Mutations includes mutators to make changes such as:
- Altering arithmetic operators (e.g.,
+
to-
) - Changing logical operators (e.g.,
&&
to||
) - Modifying string literals (e.g., turning strings into empty strings)
Setting Up Stryker for TypeScript
To get started with Stryker, run the following command in your project:
npm init stryker
This command will install Stryker and set up a configuration file tailored to your project and test framework (Mocha, Jest, Vitest, etc.).
Here is an example configuration for a Node.js project using Vitest:
{
"$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
"coverageAnalysis": "perTest",
"packageManager": "npm",
"reporters": ["html", "clear-text", "progress"],
"testRunner": "vitest"
}
Code Mutation Examples
After setting up Stryker, you can run mutation testing with the following command:
npx stryker run
This command generates a report that shows how well your test suite performs against the mutations Stryker has introduced. Let's look at an example to understand how Stryker can help improve your test quality. Consider the following simple function:
export function sayHello() {
return 'Hello';
}
A weak test might only check if the function returns a defined value:
import { sayHello } from './sayHello.js';
describe('sayHello', () => {
it('returns Hello', () => {
const text = sayHello();
expect(text).toBeDefined();
});
});
Stryker might mutate the function to return an empty string:
export function sayHello() {
return '';
}
The weak test would pass even with this mutation. A stronger test would check the actual value being returned:
import { sayHello } from './sayHello.js';
describe('sayHello', () => {
it('returns Hello', () => {
const text = sayHello();
expect(text).toBeDefined();
});
});
Identifying Unnecessary Tests
Stryker also identifies tests that do not contribute to detecting mutations. For example:
import { sayHello } from './sayHello.js';
describe('sayHello', () => {
it('returns Hello', () => {
const text = sayHello();
expect(text).toBeDefined();
});
it('tests the same', () => {
const text = sayHello();
expect(text).toBeDefined();
});
it("doesn't test anything", () => {});
});
Stryker will flag redundant tests (like the second one) and ineffective tests (like the third one).
Finding Ineffective Source Code
Consider the following TypeScript function that prints all elements of an array:
export function printArray(array: number[]) {
let index = 0;
while (index < array.length) {
console.log(array[index]);
index++;
}
return array[index - 1];
}
A mutation might alter the while loop, causing an infinite loop and a timeout. Stryker would flag this, allowing you to improve your code:
export function printArray(array: number[]) {
for (const value of array) {
console.log(value);
}
return array[array.length - 1];
}
You can then write tests to ensure your code handles these scenarios correctly:
import { printArray } from './printArray.js';
describe('printArray', () => {
it('returns the last array item', () => {
const consoleMock = vi.spyOn(console, 'log');
const lastItem = printArray([1, 2, 3, 4]);
expect(lastItem).toBe(4);
expect(consoleMock).toHaveBeenCalledTimes(4);
});
});
Disabling Mutations
We could instruct Stryker to disable certain modifications, but that would defeat the purpose of using a mutation testing framework. However, this approach can be useful if you don't have the capacity to fix all your tests at once and prefer to upgrade them gradually:
export function printArray(array: number[]) {
let index = 0;
// Stryker disable next-line BlockStatement,ConditionalExpression,EqualityOperator
while (index < array.length) {
console.log(array[index]);
// Stryker disable next-line UpdateOperator
index++;
}
return array[array.length - 1];
}
Testing Large Codebases
For large codebases, Stryker offers an incremental mode, which focuses on testing only the parts of the code that have changed since the last mutation test run. This feature is especially useful in CI environments where you may want to run tests only for new code committed in a Pull Request.