All of these compose with stack, and most are assembled for
you by Dependency.
Policies that hold state (CircuitBreaker, Bulkhead, RateLimiter,
RetryBudget, SingleFlight) are objects you construct, so their scope is
whatever you share them with. The rest (retry, timeout, hedge,
fallback) are functions that return a policy.
A schedule is a function from an attempt number to a delay. Write your own if the provided ones don't fit:
type Backoff = (attempt: number, error?: unknown) => number;
The failure is passed too, for a schedule that reads something from the error. The three provided schedules ignore it.
exponential({ base: 200, factor: 2, max: 10_000 }) // base * factor ** (attempt - 1)
constant(500) // the same delay every time
fromList([100, 500, 2_000]) // in order, repeating the last
factor defaults to 2 and max to no ceiling. fromList throws a
RangeError if the list is empty.
Every schedule is jittered, and full jitter is the default. Without jitter, clients that failed together retry together.
jitter |
Spreads the delay over |
|---|---|
"full" (default) |
[0, delay] |
"equal" |
[delay / 2, delay] |
"none" |
nothing; the delay is exact |
Pass random to make a jittered schedule exact in a test.
retryRepeats the action until it succeeds, the options say to stop, or the caller's signal aborts.
retry({
attempts: 3,
backoff: exponential({ base: 100, max: 5_000 }),
shouldRetry: (error, attempt) => isTransient(error),
budget: shared,
maxElapsed: 20_000,
});
| Option | Notes |
|---|---|
attempts |
Total, not additional. 1 never retries. |
backoff |
Required. |
shouldRetry |
Required, called with the error and the attempt that produced it. |
budget |
A RetryBudget shared with other callers. |
maxElapsed |
No attempt is started once the wait before it would carry the call past this, measured from the first attempt. |
retryAnything repeats everything, including a TypeError from a bug in your
own code.
The loop stops when the caller's signal aborts, the attempts run out,
shouldRetry returns false, a shared budget is empty, or the next backoff
would pass maxElapsed. In each case the last failure is rethrown as it was.
RetryBudgetA ceiling on retries across every call that shares it.
attempts: 3 bounds one call. If every caller is on its third attempt, the
dependency gets three times its normal load. A budget caps the total.
const budget = new RetryBudget({ ratio: 0.2, capacity: 10 });
ratio is retries earned per call: 0.2 lets the client retry a fifth of what
it sends. capacity is both the ceiling on what can be saved up and the
largest run of retries allowed when nothing has been earned yet. It starts
full, so a quiet client can still retry.
It counts calls and uses no clock or timer. Read budget.tokens for what is
left. When it runs out, a budget-exhausted event
is emitted and the failure is rethrown.
timeoutGives each attempt ms to finish, aborting that attempt's own signal with a
TimeoutError and rejecting with the same error.
timeout(2_000)
An action that watches its signal stops when the deadline passes.
This is a deadline per attempt. For a budget across all attempts, use
maxElapsed on retry. An attempt never outlives that budget: a deadline that
would run past it is cut short.
CircuitBreakerStops calling a dependency that is failing, and tries one call once the reset window has passed.
There are two trip conditions and you pick one. Consecutive failures:
const breaker = new CircuitBreaker({
threshold: 5,
resetAfter: 30_000,
});
Or a rate over a window. Use this when failures are mixed with successes: 40% of calls failing may never reach five in a row.
const breaker = new CircuitBreaker({
resetAfter: 30_000,
window: {
size: 100,
within: 60_000,
minimumCalls: 20,
failureRate: 0.5,
slowerThan: 2_000,
slowRate: 0.3,
},
});
| Window field | What it is |
|---|---|
size |
Calls remembered. |
within |
How recent a call must be to count, in ms. Without it, the window is the last size calls however old. |
minimumCalls |
Calls required before a rate can open the circuit, so two failures out of two do not read as 100%. |
failureRate |
Share of the window that must have failed, 0 to 1. |
slowerThan |
Calls at least this slow count as slow, whether or not they succeeded. |
slowRate |
Share of the window that must be slow. |
slowRate opens the circuit for an upstream that answers, but too slowly. A
circuit opened by slowness alone has no failure to report, so its
CircuitOpenError.cause is undefined.
Both forms take isFailure, which decides whether an error counts. An error it
declines is not recorded at all: it neither opens nor closes the circuit, and
passes through untouched.
probes (default 1) is how many calls are let through while half-open, and
how many must succeed to close it.
Read breaker.state ("closed", "open", "half-open"), breaker.failures
and breaker.recorded. Constructing one starts no timer.
BulkheadBounds how many calls may be in flight at once, and how many may wait, so one slow dependency cannot take every connection or worker.
const bulkhead = new Bulkhead({
concurrency: 20,
queue: 100,
queueTimeout: 1_000,
});
queue defaults to 0, which refuses instead of queueing, with a
BulkheadFullError. queueTimeout is how long a call may wait for a slot
before it is refused.
The queue is first in, first out, and a caller whose signal aborts leaves it.
adapt moves the limit: additive increase while calls succeed under pressure,
multiplicative decrease when they fail or take longer than slowerThan.
adapt: { min: 5, max: 100, slowerThan: 1_000, decrease: 0.9 }
decrease defaults to 0.9. The limit only grows while every slot is in
use. Read the current value from bulkhead.concurrency, alongside inFlight
and queued.
RateLimiterA token bucket, per process.
const limiter = new RateLimiter({
capacity: 10,
perSecond: 5,
onExhausted: "wait",
});
capacity is the bucket size, which is the largest burst allowed after an idle
period. onExhausted defaults to "wait"; "reject" refuses with a
RateLimitError carrying retryAt instead.
It refills from the clock when read, with no timer. Waiting callers are served
in the order they arrived.
Read limiter.tokens and limiter.queued, or spend a token directly with
await limiter.take(signal).
hedgeSends the same work again before the first attempt has failed, and takes the first answer.
hedge({
attempts: 2,
delay: 300,
budget: shared,
});
delay is how long to wait for the attempt in flight before starting another,
as a number or as a schedule read with the number of attempts already started.
Set it near the latency you are willing to accept, around the p95 of the call,
not the p50, or every call is sent twice.
Losing attempts are aborted as soon as one answers. The winner's signal is left alone, since its result may still be being read.
By default a hedge only handles latency: a failure goes straight to the caller,
and repeating it is retry's job. shouldHedge starts the next attempt
immediately on a failure it accepts. It has no default.
A hedge sends extra requests when the dependency is slow, so share a budget
with the retry above it. If every attempt fails, the last failure is rethrown.
SingleFlightCollapses concurrent calls that share a key into one call.
const shared = new SingleFlight();
const today = await shared.run("rates:today", readRates, signal);
It stores nothing once a call settles, so the next caller starts a new one and there is no expiry to configure.
Each caller's signal detaches only that caller. The work is aborted when the
last caller leaves. Read shared.inFlight and shared.has(key), or use
shared.policy(key) to run everything a policy wraps under one key.
fallbackAnswers with a substitute when everything else has failed: stale data, an empty list.
const rates = await fallback(() => cached)(policy(readRates))();
It goes outermost and wraps a stack (see Concepts).
shouldFallback narrows which failures get the substitute, and defaults to all
of them, since a substitute adds no load on the dependency. An aborted call
still rejects.