Rate-Limited Fan-Out
Fan-Out with Rate Limiting
When calling external APIs or databases, unbounded parallelism can overwhelm downstream systems — triggering rate limits, connection pool exhaustion, or service degradation. Rate-limited fan-out processes items in fixed-size batches, waiting for each batch to complete before starting the next. This gives you predictable resource usage while still benefiting from parallelism within each batch.
10 Items to Process
concurrency = 3
Batch 1
Item 1
Item 2
Item 3
await all ✓
Batch 2
Item 4
Item 5
Item 6
await all ✓
Batch 3
Item 7
Item 8
Item 9
await all ✓
Batch 4
Item 10
All Results
1use orcher::prelude::*;2use anyhow::Result;3use futures::stream::{self, StreamExt};45// Fan-out with concurrency limit6#[workflow(name = "rate_limited_processor", version = "1.0.0")]7pub async fn rate_limited_processor(8 ctx: WorkflowContext,9 input: BatchInput,10) -> Result<BatchResult> {11 const MAX_CONCURRENT: usize = 10;1213 println!("Processing {} items with max {} concurrent",14 input.items.len(), MAX_CONCURRENT);1516 // Fan-Out with rate limiting using buffered streams17 let results: Vec<ItemResult> = stream::iter(input.items.iter())18 .map(|item| async {19 ctx.execute_task(process_item, item.clone()).await20 })21 .buffer_unordered(MAX_CONCURRENT)22 .collect::<Vec<_>>()23 .await24 .into_iter()25 .collect::<Result<Vec<_>>>()?;2627 // Fan-In: Aggregate results28 let total_value = results.iter().map(|r| r.value).sum();2930 Ok(BatchResult {31 job_id: input.job_id,32 total_items: results.len(),33 total_value,34 results,35 })36}