Reference

Tasks


Tasks do the work that touches the outside world: API calls, database writes, file operations. A workflow schedules a task, any worker polling the workflow's task queue runs it, and the engine journals its result. A failed attempt is retried under the task's retry policy.

Tasks run at least once. The result of a task is journaled once, and replaying the workflow returns it without running the task again. But an attempt can run and then fail to report its result, for example when its worker dies, and the engine then runs it again. Write tasks so that running one twice does no harm (Idempotency).

There are two ways to define tasks:

  • Standalone tasks: one #[task] function each.
  • Task groups: several #[task] methods on a struct, under #[tasks], with shared defaults.

Standalone Tasks

1
use orcher_sdk::prelude::*;
2
3
#[derive(Serialize, Deserialize)]
4
pub struct ChargeRequest {
5
pub order_id: String,
6
pub amount_cents: u64,
7
}
8
9
#[derive(Serialize, Deserialize)]
10
pub struct Charge {
11
pub charge_id: String,
12
pub amount_cents: u64,
13
}
14
15
// Up to 3 attempts in all, 60 seconds per attempt.
16
#[task(retry = 3, timeout = 60)]
17
pub async fn charge_card(_ctx: TaskContext, request: ChargeRequest) -> Result<Charge> {
18
// Call your payment provider here.
19
Ok(Charge {
20
charge_id: format!("ch_{}", request.order_id),
21
amount_cents: request.amount_cents,
22
})
23
}

A task function is async, takes TaskContext first and exactly one input after it, and returns orcher_sdk::Result<T>. Use a struct for several values and () for none. The macro keeps the function body as <name>_impl and generates a constant with the function's name, which workflows pass to execute_task:

1
#[workflow]
2
async fn checkout(ctx: WorkflowContext, request: ChargeRequest) -> Result<String> {
3
let charge: Charge = ctx.execute_task(charge_card, request).await?;
4
Ok(charge.charge_id)
5
}

#[task] attributes

AttributeDefaultEffect
nameFunction nameThe name the task is registered and scheduled under. See the caution below.
retryMaximum attempts in all, including the first, with exponential backoff: 1 s, doubling, at most 60 s between attempts. retry = 1 means no retries.
retry_policy(max_attempts, initial_interval, max_interval, backoff_coefficient)3, 1, 60, 2.0The full policy, intervals in seconds. Takes precedence over retry.
non_retryable_errors = ["Type", ...]Error types that fail the task at once (Non-retryable errors).
timeout / timeout_mins300 sLimit on one attempt, in seconds or minutes.
heartbeat_timeoutThe engine's (60 s unless configured)Seconds without a heartbeat before the attempt is considered lost. Must be less than timeout.

A task with no retry attributes gets 3 attempts with the default backoff.

description, version, namespace, task_queue, priority, max_concurrent, rate_limit, resources(...) and memory are accepted but not applied in 0.7. A task always runs on the task queue of the workflow that schedules it, or of its session.

Retry policies and timeouts are looked up by task name on the worker that runs the workflow, when it schedules the task. If tasks and workflows are built into different binaries, link the task definitions into the workflow's worker too, or the task runs with the defaults.


Retry Policies and Errors

Non-retryable errors

1
use orcher_sdk::prelude::*;
2
use orcher_sdk::error::TaskError;
3
4
#[task(
5
retry_policy(max_attempts = 5, initial_interval = 2, max_interval = 120, backoff_coefficient = 3.0),
6
non_retryable_errors = ["CardDeclined"],
7
timeout = 30
8
)]
9
pub async fn authorize(_ctx: TaskContext, request: ChargeRequest) -> Result<String> {
10
if request.amount_cents == 0 {
11
// Listed in non_retryable_errors: fails the task on this attempt.
12
return Err(TaskError::application("CardDeclined", "amount must be positive").into());
13
}
14
if request.amount_cents > 1_000_000 {
15
// Marked non-retryable: never retried, whatever the policy lists.
16
return Err(TaskError::non_retryable("LimitExceeded", "over the card limit").into());
17
}
18
// Any other error is retried under the policy.
19
Ok(format!("auth_{}", request.order_id))
20
}
  • TaskError::application(error_type, message) fails the attempt under error_type. It is retried unless the task's non_retryable_errors lists that type.
  • TaskError::non_retryable(error_type, message) is never retried, whatever the policy says.
  • Any other error fails the attempt and is retried while attempts remain.

TaskError is in orcher_sdk::error, not the prelude. When the last attempt fails, the workflow receives WorkflowError::TaskFailed with the number of attempts made (Handling Failures).

Errors from other libraries

std::io::Error, serde_json::Error and anyhow::Error convert into orcher_sdk::Error with ?. Errors from other crates don't; map them, and pick an error type a retry policy can name:

1
#[task(retry = 3)]
2
pub async fn read_config(_ctx: TaskContext, path: String) -> Result<String> {
3
// std::io::Error and serde_json::Error convert with `?`.
4
let raw = tokio::fs::read_to_string(&path).await?;
5
6
// Other libraries' errors do not: map them, and choose the error type a
7
// retry policy can name.
8
let port: u16 = raw
9
.trim()
10
.parse()
11
.map_err(|e: std::num::ParseIntError| TaskError::non_retryable("BadConfig", e.to_string()))?;
12
Ok(format!("port {port}"))
13
}

Task Groups

Group related tasks on a struct with #[tasks]. Attributes on #[tasks(...)] are defaults for every task in the block, and a method's own #[task(...)] overrides them:

1
pub struct PaymentTasks;
2
3
#[tasks(retry = 3, timeout = 60)]
4
impl PaymentTasks {
5
/// Inherits retry = 3 and timeout = 60. Registered as "capture".
6
#[task]
7
async fn capture(_ctx: TaskContext, charge_id: String) -> Result<bool> {
8
// Capture the charge here.
9
Ok(!charge_id.is_empty())
10
}
11
12
/// Overrides retry and keeps timeout = 60. Registered as "refund".
13
#[task(retry = 10)]
14
async fn refund(_ctx: TaskContext, charge_id: String) -> Result<String> {
15
// Refund the charge here.
16
Ok(format!("re_{charge_id}"))
17
}
18
19
/// Not a task: an ordinary associated function.
20
fn provider_url() -> &'static str {
21
"https://payments.example.com"
22
}
23
}
24
25
#[workflow]
26
async fn capture_or_refund(ctx: WorkflowContext, charge_id: String) -> Result<String> {
27
let captured: bool = ctx
28
.execute_task(PaymentTasks::capture, charge_id.clone())
29
.await?;
30
if captured {
31
return Ok(format!("captured via {}", PaymentTasks::provider_url()));
32
}
33
ctx.execute_task(PaymentTasks::refund, charge_id).await
34
}

Each task is registered under its method name (or its name), not prefixed with the struct, so method names must be unique across the worker's tasks. Workflows name a grouped task through the associated constant, PaymentTasks::capture. Task methods take TaskContext first, not &self; methods without #[task] are kept as written.

#[tasks] accepts retry, timeout, timeout_mins and heartbeat_timeout as defaults that take effect, plus the same metadata attributes as #[task].


Heartbeats

The worker heartbeats every running task on its own, whatever the task's code does, so a long task needs no heartbeat code to stay alive. If its worker dies, the heartbeats stop, the engine treats the attempt as lost once heartbeat_timeout passes, and the task is retried on another worker.

ctx.heartbeat() and ctx.heartbeat_with_details(details) add a heartbeat from the task's code, folded into the worker's own: they never wait, and calling them often doesn't send more. Use heartbeat_with_details to record progress:

1
#[derive(Serialize)]
2
struct Progress {
3
done: usize,
4
total: usize,
5
}
6
7
#[task(timeout = 3600, heartbeat_timeout = 120)]
8
pub async fn import_rows(ctx: TaskContext, rows: Vec<String>) -> Result<usize> {
9
let total = rows.len();
10
for (i, row) in rows.iter().enumerate() {
11
// Write the row here.
12
let _ = row;
13
if i % 100 == 0 {
14
// Optional: the worker already heartbeats the task. This adds
15
// progress details to what it sends.
16
ctx.heartbeat_with_details(Some(Progress { done: i, total }))
17
.await?;
18
}
19
}
20
Ok(total)
21
}

ctx.heartbeat_timeout() returns the heartbeat timeout the attempt was given, if any. Both calls fail once the task has been cancelled.


Cancellation

When the engine asks a task to stop, for example because its workflow was cancelled, the worker learns of it through the task's heartbeats and cancels the task's token. Check it between units of work:

1
#[task(timeout = 3600)]
2
pub async fn reindex(ctx: TaskContext, ids: Vec<u64>) -> Result<usize> {
3
let mut done = 0;
4
for id in ids {
5
if ctx.is_cancelled() {
6
return Err(TaskError::Canceled.into());
7
}
8
// Reindex one document here.
9
let _ = id;
10
done += 1;
11
}
12
Ok(done)
13
}

Or race the work against it:

1
#[task(timeout = 3600)]
2
pub async fn wait_for_export(ctx: TaskContext, export_id: String) -> Result<String> {
3
let token = ctx.cancellation_token();
4
tokio::select! {
5
url = poll_export(&export_id) => Ok(url),
6
_ = token.cancelled() => Err(TaskError::Canceled.into()),
7
}
8
}
9
10
async fn poll_export(export_id: &str) -> String {
11
// Poll your export service until the file is ready.
12
format!("https://exports.example.com/{export_id}")
13
}

Attempts

ctx.attempt() is the attempt number, starting at 1. ctx.is_retry() is true from the second attempt on.

1
#[task(retry = 5)]
2
pub async fn call_partner(ctx: TaskContext, request_id: String) -> Result<String> {
3
// 1 on the first attempt, 2 on the first retry, and so on.
4
if ctx.is_retry() {
5
println!("attempt {} for {request_id}", ctx.attempt());
6
}
7
Ok(format!("ok {request_id}"))
8
}

Idempotency

ctx.task_id() is the same on every attempt of a task and differs between the tasks a workflow schedules, so with the workflow id it makes an idempotency key for the systems a task calls:

1
#[task(retry = 5)]
2
pub async fn send_invoice(ctx: TaskContext, customer_id: String) -> Result<String> {
3
// The same on every attempt of this task, and different for every task
4
// the workflow schedules. Pass it to the provider so a retried attempt
5
// that repeats a request the provider already handled is not applied twice.
6
let idempotency_key = format!("{}:{}", ctx.workflow_id(), ctx.task_id());
7
8
// Send the invoice here with `idempotency_key`.
9
Ok(format!("invoice for {customer_id} ({idempotency_key})"))
10
}

Don't generate a fresh id (Uuid::new_v4()) inside a task for this: each attempt would get a different one.


CPU-Heavy Work

Task code runs on the worker's Tokio runtime, which also runs its polls and heartbeats. Move CPU-heavy work to the blocking pool:

1
#[task]
2
pub async fn checksum(_ctx: TaskContext, data: Vec<u8>) -> Result<u64> {
3
// CPU-heavy work goes to the blocking pool so it does not stall the
4
// worker's polls and heartbeats.
5
let sum = tokio::task::spawn_blocking(move || data.iter().map(|&b| u64::from(b)).sum::<u64>())
6
.await
7
.map_err(|e| Error::Other(e.to_string()))?;
8
Ok(sum)
9
}

Payload Size

A task's input and result are each limited by the engine to 8 MiB by default. A result too large to send fails the task with the error type PayloadTooLarge, which is not retried. Return a reference to large data instead (Patterns).


TaskContext Reference

MethodDescription
heartbeat()Add a heartbeat from the task's code (the worker heartbeats on its own)
heartbeat_with_details(Some(details))The same, with serializable progress details
is_cancelled()Whether the engine has asked the task to stop
cancellation_token()The token that is cancelled then, for tokio::select!
attempt() / is_retry()Attempt number from 1; whether this is a retry
workflow_id() / run_id() / task_id()Identifiers; task_id() is stable across attempts
heartbeat_timeout()The heartbeat timeout this attempt was given
logger()The task's identifiers, for structured logging

Next Steps

  • Workflows: scheduling tasks and handling their failures
  • Worker: concurrency and task queues
  • Testing: mocking tasks