Guides

Best Practices & Performance


Best Practices

1. Use Durable Timers, Not Polling

Durable timers are Orcher's most resource-efficient primitive. A sleeping workflow consumes zero CPU and minimal storage — just a single database row. Never implement delays by polling in a tight loop, which wastes resources and creates unnecessary load on your task infrastructure.

1
// Good - Durable timer: zero resource consumption while waiting
2
ctx.sleep(Duration::from_secs(3600)).await?;
3
ctx.execute_task(check_status, id.clone()).await?;
4
5
// Bad - Polling wastes resources and creates unnecessary task executions
6
loop {
7
let status: Status = ctx.execute_task(check_status, id.clone()).await?;
8
if status.is_ready() {
9
break;
10
}
11
// Still running during sleep - wastes resources!
12
tokio::time::sleep(Duration::from_secs(10)).await;
13
}

2. Set Reasonable Timeouts

Every human interaction or external event wait should have a timeout. Without one, a workflow can block indefinitely if the expected signal never arrives. Choose timeouts that match your business SLA — 48 hours for an approval, 30 minutes for a payment, 7 days for document upload — and always handle the timeout case with an escalation path or default action.

1
// Good - Timeout prevents infinite wait
2
let approval: Option<ApprovalDecision> = ctx
3
.wait_for_event_with_timeout("approval", Duration::from_secs(172800))
4
.await?;
5
6
match approval {
7
Some(decision) => handle_approval(decision),
8
None => escalate_or_reject(),
9
}
10
11
// Bad - No timeout, could wait forever
12
let approval: ApprovalDecision = ctx.wait_for_event("approval").await?;

3. Track Progress with Checkpoints

For workflows that span hours or days, recording progress at key milestones gives you observability into where each workflow stands. This is invaluable for debugging, customer support, and operational dashboards. Store checkpoints as workflow state so they're queryable from outside the workflow.

1
// Good - Track progress at each milestone
2
let mut state = ProcessState::new();
3
4
state.checkpoints.push("started");
5
ctx.set_state("process_state", &state).await?;
6
7
// ... do work ...
8
9
state.checkpoints.push("halfway_complete");
10
ctx.set_state("process_state", &state).await?;

4. Handle Long Wait Failures

When a long-running wait times out, don't just fail the workflow. Build graceful degradation into your timeout handling: send a reminder, extend the deadline, escalate to a manager, or fall back to a default action. This keeps the business process moving even when humans are slow to respond.

1
// Good - Graceful handling with reminder and extended deadline
2
let payment: Option<Payment> = ctx
3
.wait_for_event_with_timeout("payment", Duration::from_secs(1800))
4
.await?;
5
6
match payment {
7
Some(payment) => process_payment(payment).await?,
8
None => {
9
// Timeout - send reminder and extend deadline
10
ctx.execute_task(send_payment_reminder, user_id.clone()).await?;
11
12
let extended_payment: Option<Payment> = ctx
13
.wait_for_event_with_timeout("payment", Duration::from_secs(3600))
14
.await?;
15
16
match extended_payment {
17
Some(p) => process_payment(p).await?,
18
None => cancel_order().await?,
19
}
20
}
21
}

5. Use Appropriate Time Scales

When writing sleep durations, prefer named constants or comments that make the intended duration immediately obvious. Raw numbers like 1209600 are hard to review and easy to get wrong. Rust's Duration::from_secs with clear comments is far more maintainable than Duration::from_millis with large numbers.

1
// Good - Clear, readable durations
2
ctx.sleep(Duration::from_secs(3600)).await?; // 1 hour
3
ctx.sleep(Duration::from_secs(86400)).await?; // 1 day
4
ctx.sleep(Duration::from_secs(604800)).await?; // 1 week
5
6
// Bad - Confusing large numbers without context
7
ctx.sleep(Duration::from_millis(3600000)).await?; // What is this?
8
ctx.sleep(Duration::from_secs(1209600)).await?; // Hard to read

6. Consider Time Zones

When your workflow needs to execute at a specific wall-clock time (e.g., "send at 9 AM Eastern"), you must compute the sleep duration relative to the target time in the correct timezone. Simply sleeping for a fixed number of hours won't account for DST transitions or timezone offsets.

1
// Good - Explicit timezone handling
2
use chrono::{DateTime, Utc, TimeZone};
3
use chrono_tz::America::New_York;
4
5
// Schedule for 9 AM Eastern Time
6
let target_time = New_York
7
.ymd(2024, 1, 15)
8
.and_hms(9, 0, 0)
9
.with_timezone(&Utc);
10
11
let now = Utc::now();
12
let wait_duration = (target_time - now).to_std()?;
13
14
ctx.sleep(wait_duration).await?;

Performance Considerations

Timer Storage

Each durable timer creates a single database entry with minimal overhead (less than 1KB). Timers are indexed for efficient firing and consume no CPU or memory while waiting. As a rule of thumb, timers are very cheap — use them freely for any delay, whether it's 5 minutes or 5 weeks.

Concurrent Long-Running Workflows

Orcher efficiently handles large numbers of concurrent long-running workflows because sleeping workflows consume almost no resources. The primary constraint is database storage for timer entries and journal rows.

Concurrent WorkflowsMemory UsageDatabase Load
1,000MinimalLow
10,000LowLow
100,000ModerateModerate
1,000,000+Carefully tuneCarefully tune

Wake-Up Latency

Timer wake-up latency depends on the timer duration and the polling interval of Orcher's internal timer subsystem. For most use cases, the latency is negligible relative to the timer duration itself.

Timer DurationWake-Up LatencyUse Case
< 1 minute±1 secondShort delays
1-60 minutes±5 secondsModerate delays
1-24 hours±30 secondsDaily tasks
> 24 hours±2 minutesLong-term processes

Common Patterns Summary

This table summarizes the most common long-running process patterns and their typical durations to help you choose the right approach for your use case.

PatternDurationUse CaseExample
Short DelayMinutesRate limiting, retriesWait 5 minutes, retry
Human ApprovalHours to DaysReviews, decisionsWait 48 hours for approval
Trial PeriodDays to WeeksFree trials, subscriptions14-day trial period
Order FulfillmentDaysMulti-stage processes3-7 day delivery
Recurring TaskHours to DaysScheduled reports, cleanupDaily at midnight
Reminder SeriesDays to WeeksUser engagementDay 1, 3, 7 reminders
Timeout PatternMinutes to HoursExternal system waits30-minute payment timeout

Real-World Use Cases

These diagrams show how long-running workflows map to real business processes that span days or weeks. Each step in the timeline corresponds to a durable timer sleep, a task execution, or an event wait in the workflow code.

Employee Onboarding

1
Onboarding Workflow (30 days)
2
├─ Day 0: Send welcome email
3
├─ Day 1: Create accounts
4
├─ Day 1: Wait for document upload (timeout: 7 days)
5
├─ Day 3: Send orientation reminder
6
├─ Day 7: First week check-in
7
├─ Day 14: Mid-month review
8
├─ Day 30: Onboarding complete survey

Subscription Lifecycle

1
Subscription Workflow (12 months)
2
├─ Day 0: Trial starts
3
├─ Day 7: Trial reminder
4
├─ Day 14: Trial expires → Convert or cancel
5
├─ Month 1-11: Monthly billing
6
├─ Day 330: Renewal reminder
7
├─ Day 365: Renewal or cancellation

Loan Application

1
Loan Processing (14-30 days)
2
├─ Day 0: Application submitted
3
├─ Day 1: Wait for documents (timeout: 7 days)
4
├─ Day 2: Credit check
5
├─ Day 3: Wait for approval (timeout: 5 days)
6
├─ Day 5: Approved → Wait for signature (timeout: 10 days)
7
├─ Day 10: Disbursement

Content Publishing Schedule

1
Content Schedule (7 days ahead)
2
├─ Day -7: Draft created
3
├─ Day -5: First review
4
├─ Day -3: Revisions
5
├─ Day -1: Final approval
6
├─ Day 0: Publish at 9 AM
7
├─ Day 0: Social media promotion
8
├─ Day 1: Performance metrics

Next Steps