Guides

Rate-Limited Fan-Out


Fan-Out with Rate Limiting

When calling external APIs or databases, unbounded parallelism can overwhelm downstream systems — triggering rate limits, connection pool exhaustion, or service degradation. Rate-limited fan-out processes items in fixed-size batches, waiting for each batch to complete before starting the next. This gives you predictable resource usage while still benefiting from parallelism within each batch.

10 Items to Process
concurrency = 3
Batch 1
Item 1
Item 2
Item 3
await all ✓
Batch 2
Item 4
Item 5
Item 6
await all ✓
Batch 3
Item 7
Item 8
Item 9
await all ✓
Batch 4
Item 10
All Results
1
use orcher::prelude::*;
2
use anyhow::Result;
3
use futures::stream::{self, StreamExt};
4
5
// Fan-out with concurrency limit
6
#[workflow(name = "rate_limited_processor", version = "1.0.0")]
7
pub async fn rate_limited_processor(
8
ctx: WorkflowContext,
9
input: BatchInput,
10
) -> Result<BatchResult> {
11
const MAX_CONCURRENT: usize = 10;
12
13
println!("Processing {} items with max {} concurrent",
14
input.items.len(), MAX_CONCURRENT);
15
16
// Fan-Out with rate limiting using buffered streams
17
let results: Vec<ItemResult> = stream::iter(input.items.iter())
18
.map(|item| async {
19
ctx.execute_task(process_item, item.clone()).await
20
})
21
.buffer_unordered(MAX_CONCURRENT)
22
.collect::<Vec<_>>()
23
.await
24
.into_iter()
25
.collect::<Result<Vec<_>>>()?;
26
27
// Fan-In: Aggregate results
28
let total_value = results.iter().map(|r| r.value).sum();
29
30
Ok(BatchResult {
31
job_id: input.job_id,
32
total_items: results.len(),
33
total_value,
34
results,
35
})
36
}

Next Steps