Implementing Mutation Testing with Jest in TypeScript

Mutation testing is a powerful technique for evaluating the quality of your test suite. It involves introducing small, deliberate changes (mutations) to your source code and then running your existing tests against these mutated versions. If your tests fail for a mutated version, it indicates that your tests are effective at catching that specific type of change. This challenge will guide you through setting up and running mutation tests using Jest and a popular mutation testing tool in a TypeScript project.

Problem Description

Your goal is to integrate mutation testing into a pre-existing TypeScript project that uses Jest for unit testing. You will be provided with a simple TypeScript function and a corresponding Jest test file. You need to configure a mutation testing tool to work with this setup and then run the mutation tests. The objective is to understand how the tool operates, identify any "weak" tests (tests that don't catch mutations), and interpret the results.

Key Requirements:

1. Setup: Install and configure a mutation testing tool compatible with Jest and TypeScript.
2. Execution: Run the mutation tests against the provided TypeScript code and its Jest tests.
3. Analysis: Interpret the mutation testing report to understand which mutations were survived by the tests and which were killed.

Expected Behavior:

Upon successful completion, you should be able to execute a command that runs the mutation tests and produces a report. This report will detail the original code, the mutations applied, whether the tests passed or failed against the mutated code, and an overall score.

Edge Cases to Consider:

• Complex Logic: While the provided example is simple, consider how mutation testing might behave with more intricate conditional logic, loops, or error handling.
• Test Coverage: Understand the relationship between your current test coverage and the effectiveness of mutation testing.

Examples

Example 1: Simple Function and Test

Source Code (math.ts):

export function add(a: number, b: number): number {
  return a + b;
}

Test Code (math.test.ts):

import { add } from './math';

describe('add', () => {
  it('should return the sum of two positive numbers', () => {
    expect(add(2, 3)).toBe(5);
  });

  it('should return the correct sum with a negative number', () => {
    expect(add(5, -2)).toBe(3);
  });

  it('should return zero when both numbers are zero', () => {
    expect(add(0, 0)).toBe(0);
  });
});

Mutation Testing Tool Command (Conceptual):

npx stryker run

Expected Output (Conceptual Report Snippet):

[Stryker Dashboard] Report generated: http://localhost:9000

Summary
-------
[1] files tested
[3] mutants tested
[1] mutants killed
[2] mutants survived

Score: 33.33%

Explanation:

A mutation testing tool like Stryker would introduce changes to math.ts. For instance, it might change a + b to a - b. The tests would then run against this mutated code. If add(2, 3) with the mutation 2 - 3 is tested and the expected result is still 5, the test fails to catch the mutation. If the expected result is correctly 3, the test kills the mutation. The report summarizes this.

Example 2: Another Mutation Scenario

Source Code (math.ts - same as above):

export function add(a: number, b: number): number {
  return a + b;
}

Test Code (math.test.ts - same as above):

import { add } from './math';

describe('add', () => {
  it('should return the sum of two positive numbers', () => {
    expect(add(2, 3)).toBe(5);
  });

  it('should return the correct sum with a negative number', () => {
    expect(add(5, -2)).toBe(3);
  });

  it('should return zero when both numbers are zero', () => {
    expect(add(0, 0)).toBe(0);
  });
});

Mutation Scenario (Conceptual):

The mutation testing tool changes return a + b; to return a * b;.

Expected Behavior of Tests:

• expect(add(2, 3)).toBe(5); will fail because 2 * 3 is 6, not 5. This mutation is killed.
• expect(add(5, -2)).toBe(3); will fail because 5 * -2 is -10, not 3. This mutation is killed.
• expect(add(0, 0)).toBe(0); will pass because 0 * 0 is 0. This mutation is survived.

Expected Output (Conceptual Report Snippet):

Summary
-------
[1] files tested
[3] mutants tested
[2] mutants killed
[1] mutants survived

Score: 66.67%

Explanation:

In this scenario, two out of three mutations are detected by the tests, leading to a higher score.

Constraints

• You must use a mutation testing tool that integrates with Jest. Stryker Mutator is a highly recommended and popular choice.
• The solution must be implemented in TypeScript.
• The provided code examples are for demonstration; you will be working with a simple, pre-defined TypeScript function and its Jest tests.
• Focus on the configuration and execution of mutation testing, not on writing complex code or extensive test suites.

Notes

• Choosing a Tool: Stryker Mutator is the de facto standard for mutation testing in JavaScript/TypeScript. You'll likely want to install @stryker-mutator/core, @stryker-mutator/jest-runner, and potentially @stryker-mutator/typescript if not using ts-jest or a similar transpiler.
• Configuration: Stryker uses a stryker.conf.json or stryker.conf.js file for configuration. You'll need to specify the test runner (Jest), the files to mutate, and any plugins required.
• Interpreting Results: A low mutation score often indicates that your tests are not comprehensive enough or are not testing the right aspects of your code. A high score means your tests are robust and likely to catch regressions.
• Performance: Mutation testing can be computationally intensive, especially for larger projects. Be aware that it may take some time to run.
• Goal: The primary goal of this challenge is to successfully set up, run, and understand the output of a mutation testing process.