Wrapping each HTTP request in a timeout does not bound the total time or load of an endpoint checker. Per-attempt timeouts, a hard deadline covering retries, retry eligibility, backoff, and the number of active endpoints need one policy. That policy must also settle ties when two limits expire at the same instant.
The example targets Rust 2024 with rustc and Cargo 1.98.1 and Tokio 1.53.1. An injected async Transport replaces external network access, while paused time exercises the policy boundaries. Its status-code list and retry count are application policy, not general HTTP operating rules.
1. Separate attempt time from the overall deadline
attempt_timeout limits one transport attempt. overall_timeout is a hard deadline covering the first attempt, every backoff, and all retries. Creating a fresh timeout per attempt can stretch total runtime with every retry, so the two limits solve different problems.
1.1. Close construction boundaries with typed configuration
Policy keeps the two timeouts, retry count, initial backoff, and maximum backoff distinct. Before any task is spawned, Checker::new returns a typed BuildError for concurrency 0, values above Semaphore::MAX_PERMITS, a zero attempt timeout, Instant + Duration overflow, or a retry exponent above 31. An overall timeout of 0 is valid: it represents a budget that is already exhausted.
pub struct Policy {
pub attempt_timeout: Duration,
pub overall_timeout: Duration,
pub max_retries: usize,
pub initial_backoff: Duration,
pub max_backoff: Duration,
}
pub enum BuildError {
ZeroConcurrency,
ConcurrencyTooLarge,
ZeroAttemptTimeout,
AttemptDeadlineOverflow,
OverallDeadlineOverflow,
RetryExponentTooLarge,
BackoffDeadlineOverflow,
}The constructor uses Instant::checked_add only as a preflight at that instant. Tokio time can advance before check runs, while it waits for a semaphore permit, or between attempts. Every runtime addition therefore remains fallible: policy deadline overflow becomes FinalOutcome::PolicyTimeOverflow, while an oversized response Retry-After remains capped by OverallDeadline.
1.2. The overall deadline wins an exact tie
Before polling an attempt, the example checks now >= deadline. It then uses a biased; select! ordered as overall deadline, attempt deadline, and transport. A transport that completes at exactly the overall deadline produces OverallDeadline; success requires completion strictly before it. If only the attempt timer and transport tie, AttemptTimeout wins.
let outcome = tokio::select! {
biased;
_ = tokio::time::sleep_until(deadline) => FinalOutcome::OverallDeadline,
_ = tokio::time::sleep_until(attempt_deadline) => FinalOutcome::AttemptTimeout,
result = &mut attempt => match result {
Ok(response) => FinalOutcome::Response(response),
Err(error) => FinalOutcome::Transport(error),
},
};timeout polls the wrapped future before checking the timeout. The tie rule here is therefore not a restatement of timeout behavior. It is a separate policy encoded by the select! branch order.
2. Limit retry eligibility by method and outcome
RFC 9110 defines a method as idempotent when multiple identical requests have the same intended effect on the server as one request. It permits automatic repetition of an idempotent request after a communication failure before a response is read, but places SHOULD NOT conditions on automatic retries for non-idempotent methods and on retrying a failed automatic retry.
2.1. Keep RFC semantics separate from application policy
The example automatically retries only Get, Put, and Delete; it does not retry Post. Its retryable outcomes are 408, 429, 500, 502, 503, 504, connection failure, and AttemptTimeout. That status set and bounded multi-retry behavior belong to this application's policy. The RFC does not mandate them. A conservative default is max_retries <= 1; larger values need explicit application-specific knowledge.
let retryable = request.method.is_idempotent()
&& attempts <= self.policy.max_retries
&& match outcome {
FinalOutcome::Response(ref response) => {
matches!(response.status, 408 | 429 | 500 | 502 | 503 | 504)
}
FinalOutcome::Transport(TransportError::Connect)
| FinalOutcome::AttemptTimeout => true,
_ => false,
};max_retries counts extra attempts after the first one. A value of 1 permits at most two attempts. In the binary example, the GET request for alpha receives 503 and then 200; the POST request for beta stops after its first 503.
2.2. Parse only Retry-After delta-seconds
RFC 9110 allows Retry-After to contain either an HTTP-date or delay-seconds. This example implements only delay-seconds = 1*DIGIT, a non-negative decimal integer number of seconds. It ignores empty values, signs, decimals, embedded whitespace, u64 overflow, and HTTP-date, then falls back to calculated backoff. HTTP-date parsing is deliberately out of scope.
pub fn parse_retry_after_delta(value: &str) -> Option<Duration> {
if value.is_empty() || !value.bytes().all(|byte| byte.is_ascii_digit()) {
return None;
}
value.parse::<u64>().ok().map(Duration::from_secs)
}Even valid delta-seconds cannot extend the hard deadline. The checker sleeps and retries only if both the chosen retry_at and the full timeout for the next attempt fit strictly before the deadline.
3. Keep backoff calculation and sleep inside one budget
Exponential backoff is initial_backoff * 2^(attempts-1), calculated with checked_mul and capped at max_backoff. Requiring max_retries <= 31 also bounds the shift exponent. If multiplication overflows, the code chooses the maximum backoff rather than wrapping to a short delay.
3.1. Never turn overflow into a fast retry
let factor = 1_u32 << (attempts - 1);
let backoff = self
.policy
.initial_backoff
.checked_mul(factor)
.unwrap_or(self.policy.max_backoff)
.min(self.policy.max_backoff);Whether the selected delay came from response delta-seconds or calculated backoff, the code checks now.checked_add(delay) again. An overflowing response delay, or a finite attempt that cannot fit completely before the deadline, returns OverallDeadline. If adding the configured attempt timeout itself overflows after time has advanced, the report instead contains PolicyTimeOverflow(AttemptDeadline). None of these paths sleeps or starts another transport call.
3.2. Hold the permit through backoff
Checker::check acquires a semaphore permit at entry and keeps _permit in scope until it returns a report. Finishing a transport attempt does not release the concurrency slot while the workflow is in backoff. One endpoint's entire retry workflow occupies one permit.
let _permit = self
.semaphore
.acquire()
.await
.expect("checker semaphore is never closed");
let started = Instant::now();
let Some(deadline) = started.checked_add(self.policy.overall_timeout) else {
return Report {
id: request.id,
attempts: 0,
outcome: FinalOutcome::PolicyTimeOverflow(
PolicyTimeOverflow::OverallDeadline,
),
retry_delays: Vec::new(),
};
};This prevents retry storms from bypassing the concurrency limit through sleeping requests. It also means overall-deadline construction happens after the wait, so constructor preflight cannot prove that runtime arithmetic still fits. A policy that limits only active transport calls has different semantics. The API contract should state exactly which span the permit covers.
4. Use JoinSet as a bounded sliding window
Spawning every input at once and making tasks wait on a semaphore limits active transports, but the number of waiting tasks still grows with the input. check_all initially spawns at most max_concurrency tasks and admits one more input whenever a tracked task finishes. Empty input returns an empty result without spawning anything.
4.1. Separate completion order from output order
JoinSet yields tasks in completion order; it is not an ordered collection. Each task carries its input index. After every successful report arrives, the checker sorts by that index, so concurrent execution still produces results in input order.
while let Some(joined) = tasks.join_next().await {
match joined {
Ok(report) => reports.push(report),
Err(error) => join_errors.push(error.to_string()),
}
if let Some((index, request)) = pending.next() {
let checker = self.clone();
tasks.spawn(async move { (index, checker.check(request).await) });
}
}The sliding window keeps the number of live tasks at or below max_concurrency. The semaphore bounds each endpoint workflow, while the spawn window prevents not-yet-started inputs from accumulating as tasks.
4.2. Drain every task after a JoinError
A JoinError from join_next is recorded rather than returned immediately. The loop keeps joining until the JoinSet is empty, sorts the error strings, and then returns CheckAllError. The caller does not observe the error while another previously started task is still cleaning up.
Tokio documents that dropping a JoinSet immediately aborts its tasks. This example does not treat drop as cleanup. A channel-and-release test confirms that every tracked task is drained before the error is returned.
5. Test boundaries with an async transport and paused time
Transport::send returns a Send future. The call that constructs that future also belongs inside the asynchronous attempt body. Synchronous setup must not run before the timeout selection begins.
5.1. Put future construction inside timed polling
let attempt = async { self.transport.send(request.clone()).await };
tokio::pin!(attempt);The scripted transport increments its call count and accesses its outcome queue inside an async move body too. With a zero overall budget, the transport is never polled and its call count remains 0. A separate regression test locks the construction boundary to timed polling.
This trait is only a minimal seam around a real HTTP client. It makes no general claim about what happens to sockets, DNS work, or connection pools when a production transport future is dropped. The result applies to the injected transport in this example and the stated policy.
5.2. Observe before advancing virtual time
Tests use #[tokio::test(start_paused = true)] and time::advance. They first observe a channel event or call count proving that an attempt started, then advance only the needed duration. No test relies on wall-clock sleep or a guess about when the scheduler will poll.
A paused Tokio runtime may automatically advance to the next timer when no other work can proceed. The tests therefore avoid claims about the detailed poll order of independently ready tasks. They assert policy-level values: deadline outcomes, call counts, and state before and after release.
6. Check the result and the policy boundary
Run these commands from the project directory.
cd examples/article-26-http-timeout-retry-concurrency
cargo fmt --all -- --check
cargo check --all-targets --all-features
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo run --quietOn Rust 1.98.1, Cargo 1.98.1, and Tokio 1.53.1, all five commands should exit with code 0. The debug and release test runs each contain 26 tests. Exact binary output is:
alpha status=200 attempts=2 retry_delays_ms=[10]
beta status=503 attempts=1 retry_delays_ms=[]
all_joined=2 max_concurrency=26.1. Contracts established by the tests
The 26 tests cover zero, upper-bound, and overflow configuration; runtime policy-time overflow after time advances and semaphore waiting; the hard overall deadline and both tie rules; idempotency-aware bounded retry; delta-seconds grammar; capped backoff; permit retention during retry; the bounded sliding window; input ordering; empty input; complete drain after JoinError; and the async transport polling boundary. Seventeen time-related tests use a paused runtime.
all_joined=2 means that both checker calls in the binary example finished. It does not mean all inputs were processed by one JoinSet. The check_all integration test separately establishes the bounded window and complete drain behavior.
6.2. Decide who owns policy before adopting it
Production code must own the choice of retryable methods and failures. This example does not cover authentication refresh, request-body replayability, side effects the server may already have applied, or the effect of dropping a real transport future. Supporting an HTTP-date in Retry-After also requires explicit clock and parser policy.
Choose the hard-deadline tie rule and permit scope before copying the implementation. This design gives the overall deadline priority, requires a complete next-attempt budget, and holds a permit for the whole workflow. Different requirements should change the branch order and permit scope first.
Full source code
The complete runnable source for this article is available in the Chapter 26 project on GitHub.
Leave a Reply