Tech Wiki

TOPICSSERIES

[Rust Zero to Production 15] Rust Closures and Iterators for Readable Data Pipelines

Suppose an endpoint monitor needs a report containing only failures and slow responses. There is no reason to build a new Vec after every step. Borrow the results with iter(), chain filter and map, then call collect only when the output really must be stored. A report that will be read once can be built directly with fold.

The first choice is ownership, not a method name. Use iter, iter_mut, or into_iter according to whether the source remains available, changes in place, or can be consumed. The closure traits follow the same idea. Fn, FnMut, and FnOnce describe what a closure does with shown values, so they are more useful as API contracts than as a hierarchy to memorize.

1. Iterator Example

The standalone Rust 2024 crate is in examples/article-15-closures-iterators. It has no external dependencies and does not use the shared endpoint-monitor crate.

[package]
name = "article-15-closures-iterators"
version = "0.1.0"
edition = "2024"
publish = false

[lints.rust]
unsafe_code = "forbid"

[lints.clippy]
all = "warn"
pedantic = "warn"

Run the complete example with these commands:

cd examples/article-15-closures-iterators
cargo fmt --check
cargo check --all-targets --all-features
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo run --quiet

Its EndpointResult owns either an HTTP status and latency for a successful check or a reason for a failed check. The pipelines below accept slices or vectors of that type.

2. The Fn Traits and Capture Behavior

A closure can borrow, mutably borrow, or own values from its environment. Rust determines which call traits it implements from what the body does with those captures.

pub fn call_twice<F>(mut operation: F, value: u64) -> (u64, u64)
where
    F: FnMut(u64) -> u64,
{
    (operation(value), operation(value))
}

pub fn visit_alerts<F>(results: &[EndpointResult], slow_ms: u64, mut visitor: F)
where
    F: FnMut(&EndpointResult),
{
    for result in results
        .iter()
        .filter(|result| result.needs_attention(slow_ms))
    {
        visitor(result);
    }
}

pub fn finish_report<F>(finish: F) -> String
where
    F: FnOnce() -> String,
{
    finish()
}

call_twice invokes the same closure twice, so FnMut is the weakest sufficient bound: it permits repeated calls while still accepting a closure that mutates captured state. The example increments a captured call counter and returns different values on the two calls, proving that this API accepts a genuinely mutating closure. A closure that implements Fn also satisfies the FnMut bound.

The visitor in visit_alerts may change an external counter on every call. The function therefore accepts its parameter as mut and requires FnMut. A closure that implements Fn can also satisfy an FnMut bound, but the reverse is not always true.

finish_report invokes its closure only once, so FnOnce is the least restrictive useful bound. In the example, move || prefix + &count.to_string() consumes the captured String named prefix through string addition. No value remains for a second call. The move keyword alone does not force a closure to be FnOnce; a move closure that merely reads its owned captures may implement the other call traits too.

An API should not demand a stronger bound than it uses. Requiring Fn for a callback that runs once needlessly rejects closures that have a valid reason to consume a capture.

3. Lazy Iterator Adapters

Adapters such as map, filter, and filter_map produce another iterator. Constructing the chain does not immediately traverse the entire input. Work starts when a consumer such as next, collect, fold, or reduce asks for items.

#[test]
fn adapters_are_lazy_until_a_consumer_asks_for_items() {
    let inspected = Cell::new(0);
    let values = [10, 20, 30];
    let mut doubled = values.iter().map(|value| {
        inspected.set(inspected.get() + 1);
        value * 2
    });

    assert_eq!(inspected.get(), 0);
    assert_eq!(doubled.next(), Some(20));
    assert_eq!(inspected.get(), 1);
}

Immediately after doubled is created, inspected is still 0. The first call to next() processes the first item and changes the count to 1. Laziness means the adapters do not automatically materialize intermediate collections, and a short consumer can stop before visiting every input. It does not prove that every iterator pipeline is fast. The closure bodies, input, and optimization profile still determine real cost.

The adapter-consumer distinction also clarifies intent. filter decides which items pass, while map changes their shape. collect creates a collection. fold accumulates items into an explicit initial value. reduce uses the first item as its initial accumulator, so it returns None for empty input.

4. Ownership Across iter Methods

The three methods may look interchangeable, but their items carry different ownership.

Choice Typical item Use the source afterward Suitable work
iter() &T Yes Inspect, filter, return borrowed views
iter_mut() &mut T After the iterator ends Update elements in place
into_iter() on Vec<T> T No; the vector is consumed Move fields, produce owned output

A function that temporarily reads alert names should use iter(). Its returned items are &str, so it does not clone the names.

pub fn alert_names(
    results: &[EndpointResult],
    slow_ms: u64,
) -> impl Iterator<Item = &str> {
    results
        .iter()
        .filter(move |result| result.needs_attention(slow_ms))
        .map(EndpointResult::name)
}

pub fn collect_alert_names(results: &[EndpointResult], slow_ms: u64) -> Vec<&str> {
    alert_names(results, slow_ms).collect()
}

alert_names returns the iterator itself. A caller that performs one pass does not need a separate Vec. collect_alert_names is an explicit alternative for a caller that must index or traverse the result repeatedly. Its vector buffer is part of the return contract, while each string remains borrowed.

Adding retry overhead to existing successful results calls for iter_mut() and a mutable reference to each item. If the result vector is no longer needed and only its owned names should survive, into_iter() is the better fit. Consuming the vector lets the code move each String out instead of cloning it.

pub fn add_retry_overhead(results: &mut [EndpointResult], overhead_ms: u64) {
    for result in results.iter_mut() {
        if let Outcome::Success { latency_ms, .. } = &mut result.outcome {
            *latency_ms += overhead_ms;
        }
    }
}

pub fn into_endpoint_names(results: Vec<EndpointResult>) -> Vec<String> {
    results
        .into_iter()
        .map(|result| result.name)
        .collect()
}

The final collect is justified because the function promises an owned collection of names. The useful target is unnecessary intermediate allocation, not allocation in the abstract.

5. Folding Endpoint Results into a Report

A report printed once does not need an intermediate Vec<String> that is immediately joined. This version borrows the results, passes only alerts, and writes each line into one String accumulator.

pub fn alert_report(results: &[EndpointResult], slow_ms: u64) -> String {
    results
        .iter()
        .filter(|result| result.needs_attention(slow_ms))
        .fold(String::new(), |mut report, result| {
            result.write_alert_line(&mut report);
            report
        })
}

pub fn average_success_latency(results: &[EndpointResult]) -> Option<u64> {
    let (total, count) = results
        .iter()
        .filter_map(EndpointResult::latency_ms)
        .fold((0_u64, 0_u64), |(total, count), latency| {
            (total + latency, count + 1)
        });

    (count > 0).then_some(total / count)
}

pub fn slowest_success(results: &[EndpointResult]) -> Option<u64> {
    results
        .iter()
        .filter_map(EndpointResult::latency_ms)
        .reduce(u64::max)
}

alert_report creates one output string. That string's internal buffer may still reallocate as it grows, and this example does not measure allocation counts or elapsed time. The narrower claim is that the pipeline does not create an intermediate Vec<String>. If report size can be predicted reliably, String::with_capacity may be worth testing. An unsupported capacity guess can merely reserve unused memory.

The average needs an explicit (total, count) starting state, which makes fold a natural fit. The slowest successful response can use the first latency as the starting value, so reduce is shorter. With no successful results, reduce returns None; the average function also returns None rather than dividing by zero.

A long pipeline does not need to be forced into one expression. Chains work well when every step is a clear transformation. If conditions share substantial state or error handling starts to dominate, named functions or an ordinary for loop will usually read better.

6. Iterator Check

Formatting, all-target checking, Clippy with warnings denied, tests, and the binary should all pass on stable rustc 1.98.1, Cargo 1.98.1, and Rust 2024. The crate has no external dependencies.

The first test run contains 7 tests. The following binary and documentation test runs each contain 0 tests.

running 7 tests
test tests::adapters_are_lazy_until_a_consumer_asks_for_items ... ok
test tests::closure_bounds_match_capture_behavior ... ok
test tests::filters_maps_and_collects_alert_names ... ok
test tests::fold_and_reduce_handle_successful_latencies ... ok
test tests::fold_builds_one_report_buffer ... ok
test tests::into_iter_moves_owned_names_without_cloning ... ok
test tests::iter_mut_updates_values_in_place ... ok

test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

This is the exact stdout from cargo run --quiet:

alerts:
auth: timeout
search: HTTP 200 in 720 ms
average successful latency: 340 ms
summary: 2 alerts
adjusted samples: (125, 125)

Closure traits expose how captured state is used, while iterator methods expose what happens to the source's ownership. Once those contracts are settled, filter, map, fold, and collect stop being decorative chaining and describe the route each value takes. In this example, collection happens only when a collection is the result; single-pass data stays in the iterator.

Full source code

The complete runnable source for this article is available in the Chapter 15 project on GitHub.

Sources


One response

  1. […] Next articleRust Closures and Iterators for Readable Data Pipelines […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech Wiki

Built with WordPress · Learn in public.