Partial Failure Handling
Partial Failure Handling
In many batch processing scenarios, you want to continue processing remaining items even when some fail. Rather than aborting the entire fan-out on the first error, you catch individual task failures, record them separately, and report both successful results and failures to the caller. This gives consumers the information they need to retry just the failed items or accept the partial result.
fan-out
Task 1
✓ ok
Task 2
✗ error
Task 3
✓ ok
Task 4
✓ ok
Task 5
✗ error
fan-in
3 Succeeded
Return results
2 Failed
Record errors
1use orcher::prelude::*;2use anyhow::Result;34#[derive(Debug, Clone, Serialize, Deserialize)]5struct ResilientResult {6 job_id: String,7 successful: Vec<ItemResult>,8 failed: Vec<FailedItem>,9 success_rate: f64,10}1112#[derive(Debug, Clone, Serialize, Deserialize)]13struct FailedItem {14 item_id: String,15 error: String,16}1718// Fan-out with partial failure handling19#[workflow(name = "resilient_processor", version = "1.0.0")]20pub async fn resilient_processor(21 ctx: WorkflowContext,22 input: BatchInput,23) -> Result<ResilientResult> {24 println!("Processing {} items with failure tolerance", input.items.len());2526 let mut successful = Vec::new();27 let mut failed = Vec::new();2829 // Fan-Out: Process all items30 let mut task_handles = Vec::new();31 for item in &input.items {32 let task = ctx.execute_task(process_item, item.clone());33 task_handles.push((item.clone(), task));34 }3536 // Fan-In: Collect results, handling failures individually37 for (item_id, task) in task_handles {38 match task.await {39 Ok(result) => {40 successful.push(result);41 }42 Err(e) => {43 println!("Item {} failed: {}", item_id, e);44 failed.push(FailedItem {45 item_id,46 error: e.to_string(),47 });48 }49 }50 }5152 let success_rate = successful.len() as f64 / input.items.len() as f64;5354 println!("Processing complete: {} successful, {} failed ({}% success rate)",55 successful.len(), failed.len(), (success_rate * 100.0) as i32);5657 Ok(ResilientResult {58 job_id: input.job_id,59 successful,60 failed,61 success_rate,62 })63}