Guides

Partial Failure Handling


Partial Failure Handling

In many batch processing scenarios, you want to continue processing remaining items even when some fail. Rather than aborting the entire fan-out on the first error, you catch individual task failures, record them separately, and report both successful results and failures to the caller. This gives consumers the information they need to retry just the failed items or accept the partial result.

fan-out
Task 1
✓ ok
Task 2
✗ error
Task 3
✓ ok
Task 4
✓ ok
Task 5
✗ error
fan-in
3 Succeeded
Return results
2 Failed
Record errors
1
use orcher::prelude::*;
2
use anyhow::Result;
3
4
#[derive(Debug, Clone, Serialize, Deserialize)]
5
struct ResilientResult {
6
job_id: String,
7
successful: Vec<ItemResult>,
8
failed: Vec<FailedItem>,
9
success_rate: f64,
10
}
11
12
#[derive(Debug, Clone, Serialize, Deserialize)]
13
struct FailedItem {
14
item_id: String,
15
error: String,
16
}
17
18
// Fan-out with partial failure handling
19
#[workflow(name = "resilient_processor", version = "1.0.0")]
20
pub async fn resilient_processor(
21
ctx: WorkflowContext,
22
input: BatchInput,
23
) -> Result<ResilientResult> {
24
println!("Processing {} items with failure tolerance", input.items.len());
25
26
let mut successful = Vec::new();
27
let mut failed = Vec::new();
28
29
// Fan-Out: Process all items
30
let mut task_handles = Vec::new();
31
for item in &input.items {
32
let task = ctx.execute_task(process_item, item.clone());
33
task_handles.push((item.clone(), task));
34
}
35
36
// Fan-In: Collect results, handling failures individually
37
for (item_id, task) in task_handles {
38
match task.await {
39
Ok(result) => {
40
successful.push(result);
41
}
42
Err(e) => {
43
println!("Item {} failed: {}", item_id, e);
44
failed.push(FailedItem {
45
item_id,
46
error: e.to_string(),
47
});
48
}
49
}
50
}
51
52
let success_rate = successful.len() as f64 / input.items.len() as f64;
53
54
println!("Processing complete: {} successful, {} failed ({}% success rate)",
55
successful.len(), failed.len(), (success_rate * 100.0) as i32);
56
57
Ok(ResilientResult {
58
job_id: input.job_id,
59
successful,
60
failed,
61
success_rate,
62
})
63
}

Next Steps