Mutation Testing in C# with Stryker.NET

The coverage report says 100 percent. Every line in this price function ran, both branches ran, and the test suite is green:

public static decimal Price(decimal subtotal, bool preferred) =>
    preferred ? subtotal * 0.90m : subtotal;

[Theory]
[InlineData(100, true)]
[InlineData(100, false)]
public void Price_is_positive(decimal subtotal, bool preferred)
{
    var result = Price(subtotal, preferred);

    Assert.True(result > 0);
}

Now change the preferred-customer multiplier from 0.90m to 1.00m. The discount disappears, the customer pays too much, and every test still passes. Coverage told the truth: the test executed the line. It never promised that an assertion could detect the wrong behavior on that line.

Mutation testing asks the missing question. A tool makes a small, controlled change to production code, runs the tests, and watches whether the suite turns red. If a test fails, the mutant is killed. If the suite stays green, the mutant survives and exposes a gap worth investigating. The official Stryker.NET introduction describes it as temporarily inserting bugs to test your tests. That phrasing is not marketing poetry. It is the entire feedback loop.

Test the tests

One covered line, two very different signals

line coverage: 100%

Stryker changes the production code

subtotal * 0.90m → subtotal * 1.00m

Weak assertion

Assert.True(result > 0)

test stays green mutant survived

Behavior assertion

Assert.Equal(90m, result)

test turns red mutant killed

Coverage proves code was visited. Mutation testing provides evidence that the visit contained an observation strong enough to notice a plausible bug.

What Stryker.NET actually does

Stryker parses the project and creates variants of expressions that often carry behavior: change < to <=, replace + with -, invert a Boolean, remove a method call, or alter a constant. It then selects relevant tests, compiles the mutated program, and runs the suite. The production source on disk is not rewritten and left broken. Mutants live inside the mutation run.

The outcomes are more useful than a single percentage:

Outcome	What happened	What to do
Killed	At least one test failed against the changed code	Keep the test; it observed the behavior
Survived	All selected tests stayed green	Inspect the mutant and add or improve a behavior assertion
No coverage	No test reached the mutated statement	Decide whether the code or the missing test is the problem
Timeout	The mutant made the run exceed its limit	Inspect loops, waits, and the timeout rather than counting it as confidence
Ignored	Configuration deliberately excluded the mutation	Keep the reason narrow and reviewable

The mutation score is normally killed mutants divided by the mutants that received a meaningful result. It is a summary, not the work queue. A project at 74 percent with every payment invariant protected may be safer than one at 92 percent that kills punctuation changes and misses a boundary in authorization code.

Run the first mutation test locally

Use a local tool manifest so the team and CI restore the same tool version. The current Stryker.NET getting-started guide lists its runtime prerequisite and the latest commands; the project being mutated may target an earlier supported framework.

dotnet new tool-manifest
dotnet tool install dotnet-stryker
dotnet tool restore

cd tests/Store.Domain.Tests
dotnet stryker

For a small test project, that is enough. Stryker discovers the referenced source project, runs the ordinary suite once, creates mutants, and writes a report under StrykerOutput. Commit the tool manifest, not the generated report.

Larger solutions should make the scope explicit in stryker-config.json. This example mutates one domain project, excludes generated code, emits a readable report, and fails CI below an illustrative 60 percent floor:

{
  "stryker-config": {
    "solution": "../Store.sln",
    "project": "Store.Domain.csproj",
    "mutate": [
      "Pricing/**/*.cs",
      "!**/*.Generated.cs"
    ],
    "reporters": ["progress", "html"],
    "thresholds": {
      "high": 80,
      "low": 60,
      "break": 60
    }
  }
}

Do not copy the threshold because sixty sounds respectable. Run the suite, triage the survivors, and set the first break threshold just below the reviewed baseline. Ratchet it upward when valuable mutants are killed. Stryker's current configuration reference also supports mutate globs, changed-code runs with since, and reusable baselines. Those controls make a mutation gate affordable; they should not hide business logic merely because it is inconvenient to test.

Triage a survivor before writing another test

A surviving mutant is a question, not an automatic defect. Read it in this order.

First, can the mutation change observable behavior? Suppose Stryker changes an exception message, but the message is internal telemetry that no consumer relies on. An assertion against the exact text would kill the mutant and make the suite more brittle without protecting a contract. Marking a specific mutation or method as ignored can be honest when the behavior is deliberately irrelevant. Ignoring every string mutation globally because the report is noisy is rarely honest.

Second, is the changed behavior part of a real promise? A boundary change from age >= 18 to age > 18 plainly changes who is eligible. A removed logging call may or may not matter, depending on whether an audit record is a compliance requirement or a debug convenience. The codebase, not the mutation operator, defines the contract.

Third, write the smallest test that distinguishes the correct implementation from the mutant. For the discount example, assert the outcome the business promises:

[Fact]
public void Preferred_customer_receives_ten_percent_discount()
{
    var result = Price(subtotal: 100m, preferred: true);

    Assert.Equal(90m, result);
}

That test kills the 1.00m mutant for the right reason. It survives refactoring because it knows nothing about branches or multipliers; it states the customer-visible behavior. The discipline is the same one behind unit testing that survives refactoring: assert the promise, not the current choreography.

Finally, ask whether the production code should be simpler. A forest of equivalent mutants often points at redundant conditions, unreachable branches, or defensive code whose contract nobody can state. Deleting behaviorless code can improve the mutation score without adding a test, and that is not gaming the number. It is removing places where future readers expected meaning and found none.

Equivalent mutants are a limit, not an excuse

Some mutations compile to code that is observably identical for every valid input. Consider a comparison changed at a boundary that an earlier validation has already excluded. No test can kill that mutant through the public contract because there is no behavioral difference to observe.

Proving equivalence in the general case is hard, so tools cannot label every equivalent mutant for you. Review the code path, document a narrow suppression if the result is genuinely equivalent, and move on. Do not weaken production encapsulation or assert private details to satisfy a dashboard.

The opposite mistake is declaring an inconvenient survivor equivalent after trying two inputs. If the change affects a return value, state transition, exception type, external call, or security decision, assume there is a missing observation until you can show otherwise. Boundary values are especially fertile: zero, empty, exactly-at-the-limit, and one step either side.

Microsoft's current mutation testing guidance for .NET makes the same practical point: do not chase 100 percent; focus on high-risk and business-critical areas. The useful target is reviewed survivors, not a perfect color.

Put mutation testing in CI without making CI unbearable

Mutation testing compiles and runs far more than an ordinary test job, so a whole-solution run on every commit is often the fastest way to make the team delete it. Start with code where a silent wrong answer is expensive and tests are already fast: pricing rules, permission decisions, parsers, state machines, and pure domain calculations.

Use three layers:

1. Run a narrow changed-code mutation set on pull requests. Fail only when the reviewed score drops below the baseline or a new, relevant survivor appears.
2. Run a broader project-level set on a schedule, when a ten-minute feedback loop does not block a developer waiting to push.
3. Review exclusions like code. Every ignored file or mutation type needs a reason that remains true.

The threshold should prevent regression, not create a cleanup emergency in every old module. A ratchet protects the confidence already earned while leaving room to improve one survivor at a time. That is more durable than declaring 80 percent on Friday and adding hollow assertions until the build turns green.

Practice the feedback loop from the other side

Most mutation tools change production code and ask whether an existing suite catches the change. Katabench's test-writing exercises turn the same model into deliberate practice. You receive a correct subject and write the tests. The grader runs your suite against that subject, where every test must pass, and then against authored planted-bug mutants, where enough tests must fail. A suite that merely executes every method earns nothing if the bugs survive.

The test-writing track makes the signal concrete without revealing the hidden bug, and the grading model reports caught and surviving mutants separately. That is not a replacement for Stryker in a production repository. It is a focused way to practice the judgment Stryker demands: identify the contract, choose the boundary, and write an assertion that notices the wrong behavior for the right reason.

Once that judgment becomes familiar, a surviving mutant stops looking like a score deduction. It becomes a precise question from the codebase: "What did you believe this test proved?"