Circuit Breaker Pattern in .NET with Polly

Your catalog service is down. The first request waits through three retries and fails after twelve seconds. The second does the same. So do the next hundred requests, even though the caller has enough evidence to predict the answer.

A retry policy is now multiplying an outage. Every caller occupies a connection, spends its latency budget, and adds traffic to a dependency that may already be overloaded. The circuit breaker pattern changes that decision. After enough recent failures, the caller stops making the remote call and fails locally for a while. It later permits a probe to discover whether the dependency recovered.

That sounds simple until the configuration asks four different questions: how many calls count as evidence, which results count as failures, how long the evidence remains relevant, and how long to wait before probing. A production circuit breaker is those decisions, not the fluent API around them.

Circuit state machine

Failure evidence changes where the call stops

caller-side decision

Closed

calls flow

sample outcomes from the dependency

→

Open

fail locally

do not spend another network timeout

→

Half-open

probe once

let fresh evidence decide recovery

Closed → Open: threshold crossed

Open → Half-open: break duration elapsed

Probe: close on success, reopen on failure

A timeout limits the cost of one attempt. A retry spends more attempts. A circuit breaker decides when recent evidence says another attempt should not start.

What the three circuit breaker states mean

In the closed state, calls reach the dependency normally. The breaker observes only the outcomes its predicate handles, such as connection failures, timeouts, or selected HTTP responses. A closed circuit does not mean the dependency is healthy. It means the evidence has not crossed the configured threshold.

When the threshold is crossed, the circuit becomes open. New executions are rejected locally with Polly's BrokenCircuitException; the dependency is not called. The original failure still reaches the request that caused the transition. The breaker does not swallow failures, retry them, or manufacture a fallback response.

After the break duration, the next eligible call moves the circuit to half-open and acts as a probe. A successful probe closes the circuit. A handled failure opens it again. Nothing changes in a background timer by itself; a later execution supplies the evidence that recovery happened.

The current Polly circuit breaker documentation describes this as sampled failure detection followed by local short-circuiting. That distinction is important. A breaker is a local opinion based on what this process recently observed, not a global health oracle.

Configure the current .NET HTTP resilience stack

For outbound HTTP in modern .NET, start with the Microsoft.Extensions.Http.Resilience package. It integrates Polly resilience strategies into IHttpClientFactory. Microsoft's current HTTP resilience guide documents the standard handler and its timeout, retry, circuit-breaker, and rate-limiter pipeline.

This named client uses the standard handler and makes its breaker choices explicit:

using Polly;

builder.Services
    .AddHttpClient("catalog", client =>
        client.BaseAddress = new Uri("https://catalog.internal"))
    .AddStandardResilienceHandler(options =>
    {
        options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(12);
        options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(3);

        options.Retry.MaxRetryAttempts = 3;
        options.Retry.Delay = TimeSpan.FromMilliseconds(300);
        options.Retry.BackoffType = DelayBackoffType.Exponential;
        options.Retry.UseJitter = true;

        options.CircuitBreaker.MinimumThroughput = 8;
        options.CircuitBreaker.FailureRatio = 0.5;
        options.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(10);
        options.CircuitBreaker.BreakDuration = TimeSpan.FromSeconds(15);
    });

The configuration says: once at least eight observed executions exist inside a ten-second window, open when half of them are handled failures, then refuse calls for fifteen seconds before allowing a probe. If only seven calls happen in the window, the breaker stays closed even if all seven fail. MinimumThroughput protects low-traffic dependencies from opening on a tiny sample.

The standard handler already has defaults. Writing the values is still useful when they encode the service's traffic and recovery expectations, but copying these example numbers is not tuning. Eight executions may be one second of traffic in production and a full day in an internal admin service.

Choose failures that say the dependency is unhealthy

Do not count every unsuccessful business response. A 400 Bad Request means the caller sent something invalid. A 401 Unauthorized means authentication failed. Opening a circuit after those responses hides a caller defect behind a resilience mechanism and blocks unrelated valid requests.

The standard HTTP handler treats common transient evidence such as HttpRequestException, attempt timeouts, HTTP 408, HTTP 429, and server errors as handled outcomes. Review that set against the actual contract. A payment provider may use a specific response to report a permanent decline; a legacy service may return 500 for validation mistakes. The status code alone cannot repair a bad protocol.

Cancellation deserves separate treatment. A request canceled because its caller went away is not proof that the dependency is unhealthy. Keep cancellation flowing through the pipeline and avoid turning expected caller cancellation into failure evidence.

Tune the four numbers as one decision

MinimumThroughput answers whether the sample is large enough to trust. Set it above random noise but below the number of calls an outage can safely consume before the caller starts refusing work.

FailureRatio answers how unhealthy the sample must be. A value of 0.5 means at least half of the handled sample failed. Lower ratios react faster but can open during ordinary error bursts. Higher ratios tolerate more wasted calls during a real incident.

SamplingDuration defines how recent the evidence must be. It should be long enough to collect the minimum throughput during normal traffic and short enough that yesterday's failure burst does not describe the dependency now.

BreakDuration is not a punishment period. It is a recovery guess. A five-second break may suit an overloaded service that needs breathing room; a dependency with a known thirty-second restart time needs a different guess. Polly also supports a generated break duration when the recovery interval should respond to failure evidence.

Write the intended behavior before choosing numbers:

At normal traffic, eight attempts arrive within ten seconds.
If four fail, stop calling for fifteen seconds.
Then allow fresh evidence to close or reopen the circuit.

That statement is reviewable. Four unexplained numeric properties are not.

Retry and circuit breaker solve different failures

Retry is for faults likely to disappear on another attempt. The breaker is for evidence that attempts are no longer worth starting. Combining them is common, but composition changes the evidence.

In the standard HTTP pipeline, retry wraps the circuit breaker. Individual attempts pass through the breaker, so one logical request with three retries can contribute several failures to the sample. That can be exactly right: the dependency received several failed attempts. It also means a threshold described as "eight customer requests" would be wrong. Count what the pipeline actually observes.

Never blindly retry unsafe operations. A timed-out POST may have completed on the server before its response was lost, and another attempt can duplicate the side effect. Disable retries for unsafe HTTP methods unless the operation has an idempotency guarantee. The breaker can still observe the single attempt and protect later calls.

The other strategies have separate jobs:

Strategy	Decision it owns
Attempt timeout	How long one network attempt may take
Retry	Whether another attempt is safe and likely to help
Circuit breaker	Whether recent evidence says attempts should start at all
Concurrency limiter	How many callers may occupy the dependency at once
Fallback	What honest degraded result, if any, the caller may return

A cached product list can sometimes be labeled stale and returned as a fallback. A payment approval cannot be invented because the circuit is open. Resilience includes refusing honestly.

Scope the breaker to the failure boundary

A single global circuit for every outbound call lets one broken service block healthy ones. At the other extreme, creating a new breaker for every request erases the shared failure history and it will never gather enough evidence to open.

Use a named or typed HttpClient for a stable dependency boundary. If one client calls multiple authorities, partition the resilience pipeline by authority so failures from one host do not open the others. Polly's current guidance recommends AddResilienceHandler with SelectPipelineByAuthority() for that case.

Think carefully before separating every endpoint on the same service. Independent breakers prevent a broken report endpoint from blocking a healthy catalog endpoint, but they also fragment evidence when the real failure is shared DNS, networking, or process availability. Scope should match the failure mode you intend to contain.

Test transitions with a real failure

A unit test that asserts four option values proves configuration, not behavior. Exercise the state machine from outside the caller:

1. Make the dependency fail reliably and send enough traffic to satisfy minimum throughput.
2. Record the time and dependency-side attempt count for each call.
3. Verify later calls fail quickly without increasing the dependency's count.
4. Restore the dependency and wait beyond the break duration.
5. Send another call and verify the probe reaches the dependency and closes the circuit on success.

Also emit the transition telemetry. Polly reports circuit-opened, half-opened, and closed events, and BrokenCircuitException.RetryAfter can describe how long the local refusal is expected to continue. Alerting on every rejected request creates a storm after the useful event already happened. Alert on state transitions and sustained open duration; count rejections as impact.

Katabench's resilience labs make this failure loop concrete. You stop a real Redis or Postgres container, call through a .NET gateway, watch the first requests spend their budgets, and then see an open breaker refuse locally without touching the dependency. After the container returns, the probe closes the circuit without restarting the caller. The hands-on .NET labs and lab documentation cover the environment and workflow.

That experiment builds the judgment the configuration needs. The goal is not to add a circuit breaker everywhere. It is to identify a remote failure boundary, define what counts as evidence, and prove the caller stops making an outage more expensive.