Hone logo
Hone
Problems

Jest Mutation Score Calculator

Mutation testing is a powerful technique for assessing the quality of your unit tests. It works by introducing small, deliberate changes (mutations) to your source code and verifying that your tests fail when these mutations are present. A high mutation score indicates that your tests are effectively catching errors and providing good coverage. This challenge asks you to create a Jest-based tool that calculates a mutation score for a given TypeScript file.

Problem Description

You need to build a TypeScript utility that analyzes a given TypeScript file, introduces a set of predefined mutations, and then runs Jest tests against the mutated code. The utility should then calculate and return a mutation score, representing the percentage of mutations that were "killed" (i.e., caused a test failure). A mutation is considered "killed" if at least one test fails after the mutation is applied.

Key Requirements:

  • Mutation Generation: The utility should be able to generate a set of simple mutations. For this challenge, focus on these mutation types:
    • Arithmetic Operator Replacement: Replace + with -, * with /, and / with * in arithmetic expressions.
    • Boolean Operator Replacement: Replace && with || and || with &&.
    • Comparison Operator Replacement: Replace == with !=, != with ==, > with <, and < with >.
  • Code Modification: The utility must be able to modify the original TypeScript file in-memory based on the generated mutations. Do not write the changes to disk.
  • Jest Execution: The utility should execute Jest tests against the mutated code.
  • Mutation Score Calculation: The utility should calculate the mutation score as: (Number of Killed Mutations / Total Number of Mutations) * 100.
  • Error Handling: The utility should gracefully handle cases where Jest tests fail due to syntax errors in the mutated code. These should not be counted as killed mutations.

Expected Behavior:

The utility should take a TypeScript file path as input and return a number representing the mutation score. The utility should run Jest tests and accurately determine which mutations "killed" the tests.

Edge Cases to Consider:

  • Empty File: Handle the case where the input file is empty.
  • No Mutations Found: Handle the case where no mutations are found in the file.
  • Syntax Errors After Mutation: Jest might fail due to syntax errors introduced by the mutation. These should not be counted as killed mutations.
  • Complex Expressions: The mutations should be applied correctly even within complex expressions.
  • Comments: Ignore comments when applying mutations.
  • String Literals: Do not attempt to mutate string literals.

Examples

Example 1:

Input: A TypeScript file containing: `const sum = 1 + 2;` and a Jest test that asserts `sum === 3`.
Output: 100
Explanation: The mutation replacing `+` with `-` will cause the Jest test to fail, resulting in a mutation score of 100%.

Example 2:

Input: A TypeScript file containing: `const product = 5 * 5;` and a Jest test that asserts `product === 25`.
Output: 100
Explanation: The mutation replacing `*` with `/` will cause the Jest test to fail, resulting in a mutation score of 100%.

Example 3:

Input: A TypeScript file containing: `const isEqual = a == b;` and a Jest test that asserts `isEqual === true` when `a` and `b` are equal.
Output: 100
Explanation: The mutation replacing `==` with `!=` will cause the Jest test to fail when `a` and `b` are equal, resulting in a mutation score of 100%.

Constraints

  • File Size: The input TypeScript file should be no larger than 10KB.
  • Mutation Types: Only the specified mutation types (Arithmetic, Boolean, Comparison) are required.
  • Performance: The execution time should be reasonable (under 5 seconds for typical files).
  • Dependencies: You can use Jest and any other reasonable TypeScript libraries.
  • Mutation Count: The total number of mutations generated should be limited to a maximum of 20 to prevent excessive test execution time.

Notes

  • Consider using a regular expression-based approach to identify and replace the target operators.
  • You'll need to use the Jest API to run tests programmatically. Look into jest.run or similar functions.
  • Focus on the core logic of mutation generation and score calculation. Error handling and edge case coverage are important but can be simplified for this challenge.
  • The goal is to demonstrate an understanding of mutation testing principles and the ability to integrate with Jest.
  • You do not need to create a command-line interface or package this as a standalone tool. A function that takes a file path and returns the mutation score is sufficient.
Loading editor...
typescript