Reference

Workflows


A workflow is an async function that orchestrates tasks, timers, events and other workflows. The engine journals the outcome of every step. When a workflow resumes, after a timer fires, a task completes or a worker restarts, the worker runs the function again from the top and each step that already has a journaled outcome returns it instead of running again. This is replay, and it is why workflow code must be deterministic: given the same journal, it must issue the same steps in the same order.

  • Do side effects (network, files, databases) in tasks or ctx.execute, never directly in the workflow.
  • Read time and randomness from ctx.time() and ctx.rand(), not the system clock or a thread RNG.
  • Always return the error from a step with ?, or match it and pass on what you don't handle (see Handling Failures).

Defining a Workflow

1
use orcher_sdk::prelude::*;
2
3
#[derive(Serialize, Deserialize)]
4
pub struct OrderInput {
5
pub order_id: String,
6
pub amount_cents: u64,
7
}
8
9
#[derive(Serialize, Deserialize)]
10
pub struct OrderOutput {
11
pub order_id: String,
12
pub charge_id: String,
13
}
14
15
#[task(retry = 3)]
16
async fn charge_card(_ctx: TaskContext, input: OrderInput) -> Result<String> {
17
// Call your payment provider here.
18
Ok(format!("ch_{}", input.order_id))
19
}
20
21
// The workflow type is "process_order"; without `name` it would be the
22
// function name, which here is the same.
23
#[workflow(name = "process_order")]
24
async fn process_order(ctx: WorkflowContext, input: OrderInput) -> Result<OrderOutput> {
25
let order_id = input.order_id.clone();
26
let charge_id: String = ctx.execute_task(charge_card, input).await?;
27
Ok(OrderOutput {
28
order_id,
29
charge_id,
30
})
31
}

A workflow function is async, takes WorkflowContext first and exactly one input after it, and returns orcher_sdk::Result<T>. The input and output are serialized as JSON, so they need Serialize and Deserialize. Use a struct when a workflow needs several values, and () when it needs none:

1
#[workflow]
2
async fn nightly_report(ctx: WorkflowContext, _input: ()) -> Result<u64> {
3
let rows: u64 = ctx.execute_task(count_rows, ()).await?;
4
Ok(rows)
5
}
6
7
#[task]
8
async fn count_rows(_ctx: TaskContext, _input: ()) -> Result<u64> {
9
// Query your database here.
10
Ok(42)
11
}

#[workflow] attributes

AttributeEffect
nameThe workflow type that clients start. Defaults to the function name.
timeout, versionNo effect; each produces a deprecation warning. Set the deadline per start with StartWorkflowOptions::with_workflow_execution_timeout (Client), and the code release on the worker with WorkerBuilder::version_id or ORCHER_VERSION_ID (Worker).
namespaceA compile error. A workflow runs in whatever namespace its worker serves; set the namespace on the worker and where workflows are started.

The macro also accepts description, task_queue, max_concurrent, tags(...), retry_policy(...), cron, schedule and enabled, but in 0.7 the worker does not apply them. Choose the task queue, retry policy and cron schedule when you start the workflow, with StartWorkflowOptions (Client).


Executing Tasks

ctx.execute_task(task, input) schedules a task and waits for its result. Name the task with the constant #[task] generates, or with a string:

1
#[workflow]
2
async fn charge_by_name(ctx: WorkflowContext, input: OrderInput) -> Result<String> {
3
// A typed reference: a misspelled or renamed task fails to compile.
4
// ctx.execute_task(charge_card, input)
5
// A string name: checked only when the workflow runs. Useful for a task
6
// that is registered by a worker built from another crate.
7
let charge_id: String = ctx.execute_task("charge_card", input).await?;
8
Ok(charge_id)
9
}

The typed reference only checks the task's name. The input and output are serialized as JSON and checked when the task runs, so the output type you ask for (let x: T = ...) must match what the task returns. Name the output type even when it is (): execute_task is generic over it.

Tasks run on the workflow's task queue. A task's retry policy and timeouts come from its #[task] attributes, which the worker running the workflow reads, so link the task's crate into that worker. A task scheduled by string name that the worker doesn't know runs with the defaults: 3 attempts and a 300-second timeout. See Tasks.

Inline steps

For a one-off side effect that doesn't need its own task, ctx.execute(step_name, closure) runs an async closure on the workflow's worker and journals its result:

1
#[workflow]
2
async fn tag_order(ctx: WorkflowContext, order_id: String) -> Result<String> {
3
// Runs on this worker, inside the activation, and journals the result.
4
// On replay the journaled value is returned and the closure is skipped.
5
let label: String = ctx
6
.execute("make_label", move || async move {
7
// A one-off side effect, for example a call to an internal service.
8
Ok(format!("label-{order_id}"))
9
})
10
.await?;
11
Ok(label)
12
}
execute_task(task, input)execute("name", closure)
Runs onAny worker polling the queueThe worker running the workflow, during the activation
RetriesUnder the task's retry policyNone; an error is returned to the workflow
TimeoutThe task's timeout (300 s by default)None
Reuse and testingShared by workflows, mockable in testsLocal to the workflow

Both are at least once: a step that ran but whose result was never journaled, because a worker died at the wrong moment, runs again. Make side effects idempotent (Tasks).


Timers

1
#[workflow]
2
async fn send_reminder(ctx: WorkflowContext, user_id: String) -> Result<()> {
3
// A durable timer: the engine fires it, so it survives worker restarts.
4
ctx.sleep(std::time::Duration::from_secs(24 * 60 * 60)).await?;
5
6
// The same, with an id you choose instead of one taken from the step order.
7
ctx.sleep_with_id("second_reminder", std::time::Duration::from_secs(60 * 60))
8
.await?;
9
10
ctx.execute_task(notify_user, user_id).await
11
}
12
13
#[task]
14
async fn notify_user(_ctx: TaskContext, user_id: String) -> Result<()> {
15
// Send the notification here.
16
let _ = user_id;
17
Ok(())
18
}

ctx.sleep starts a durable timer that the engine fires, so it holds across worker restarts and no worker is busy while it waits. Its id comes from the step's position in the code; sleep_with_id lets you choose it.


Child Workflows

1
use orcher_sdk::workflow::OrphanPolicy;
2
3
#[derive(Serialize, Deserialize)]
4
pub struct ShipmentInput {
5
pub order_id: String,
6
}
7
8
#[workflow]
9
async fn ship_order(_ctx: WorkflowContext, input: ShipmentInput) -> Result<String> {
10
Ok(format!("shipped {}", input.order_id))
11
}
12
13
#[workflow]
14
async fn fulfill_order(ctx: WorkflowContext, order_id: String) -> Result<String> {
15
// Start the child and wait for its result.
16
let shipped: String = ctx
17
.execute_child_workflow(
18
"ship_order",
19
ShipmentInput {
20
order_id: order_id.clone(),
21
},
22
)
23
.await?;
24
25
// Or start it with options, keep the handle, and wait later.
26
let options = ChildWorkflowOptions::new()
27
// Derive the child's id from the parent's so it is the same on replay
28
// and unique per parent.
29
.workflow_id(format!("{}-invoice", ctx.workflow_id()))
30
.execution_timeout(std::time::Duration::from_secs(600))
31
// Leave the child running if the parent closes first.
32
.orphan_policy(OrphanPolicy::Abandon);
33
let invoice = ctx
34
.start_child_workflow_with_options("send_invoice", order_id, options)
35
.await?;
36
let invoice_id: String = invoice.result().await?;
37
38
Ok(format!("{shipped}; invoice {invoice_id}"))
39
}
  • execute_child_workflow(type, input) starts a child and waits for its result. start_child_workflow(type, input) returns a ChildWorkflowHandle straight away, so you can start several children before waiting on any of them; handle.result() waits, and the handle also has send_event and cancel.
  • ChildWorkflowOptions sets the child's workflow_id, execution_timeout, task_queue (the parent's by default) and orphan_policy: what happens to the child when the parent closes first. OrphanPolicy::Cancel (the default) asks the child to cancel, Terminate ends it at once, and Abandon leaves it running.
  • Without a workflow_id, the child's id is derived from its position in the parent's code. Derive it from ctx.workflow_id() as above when you want it to be unique and recognizable.
  • A child that fails, is cancelled, terminated or times out surfaces in the parent as WorkflowError::ChildWorkflowFailed.

Events

Clients send events to a running workflow with handle.send_event(name, payload) (Client). The workflow receives them by name:

1
#[derive(Serialize, Deserialize)]
2
pub struct Approval {
3
pub approved: bool,
4
pub approver: String,
5
}
6
7
#[workflow]
8
async fn expense_approval(ctx: WorkflowContext, expense_id: String) -> Result<String> {
9
// Suspends until a client sends an "approval" event to this workflow.
10
let first: Approval = ctx.wait_for_event("approval").await?;
11
if !first.approved {
12
return Ok(format!("{expense_id} rejected by {}", first.approver));
13
}
14
15
// Waits at most 24 hours. The deadline is a durable timer; `None` means it
16
// fired before the event arrived.
17
let second: Option<Approval> = ctx
18
.wait_for_event_with_timeout("approval", std::time::Duration::from_secs(24 * 60 * 60))
19
.await?;
20
21
Ok(match second {
22
Some(a) if a.approved => format!("{expense_id} approved"),
23
Some(a) => format!("{expense_id} rejected by {}", a.approver),
24
None => format!("{expense_id} expired"),
25
})
26
}
  • Events of one name are delivered oldest first. An event that arrives before the workflow waits for it is kept until it does.
  • wait_for_event_with_timeout returns Ok(None) when the timeout passes first. The deadline is a durable timer, so it holds while no worker is running the workflow. An event that arrives after the deadline is kept for the next wait on that name.

Handling Failures

A task that fails for good, because its attempts are used up or it failed with a non-retryable error, is returned to the workflow as Error::Workflow(WorkflowError::TaskFailed { task_type, attempts, reason, .. }). The workflow can handle it or return it, which fails the workflow:

1
use orcher_sdk::error::WorkflowError;
2
3
#[task(retry = 3)]
4
async fn reserve_stock(_ctx: TaskContext, order_id: String) -> Result<String> {
5
// Call your inventory service here.
6
Ok(format!("res_{order_id}"))
7
}
8
9
#[workflow]
10
async fn reserve_or_backorder(ctx: WorkflowContext, order_id: String) -> Result<String> {
11
match ctx.execute_task(reserve_stock, order_id.clone()).await {
12
Ok(reservation) => Ok(reservation),
13
// The task failed for good: its retries are used up, or it failed with
14
// a non-retryable error.
15
Err(Error::Workflow(WorkflowError::TaskFailed {
16
attempts, reason, ..
17
})) => Ok(format!(
18
"{order_id} backordered after {attempts} attempts: {reason}"
19
)),
20
// Everything else, including the suspension signal the SDK uses while
21
// the task is still running, must be returned unchanged.
22
Err(other) => Err(other),
23
}
24
}

A workflow that returns an error fails. It is not retried unless it was started with a retry policy (Client).


Queries, Updates and Workflow State

ctx.set_state and ctx.get_state keep JSON values by key for the current run. The state is rebuilt by replay, like everything else in the workflow, so it is a convenient place for values that query handlers report.

1
#[derive(Serialize, Deserialize)]
2
pub struct Address {
3
pub street: String,
4
pub city: String,
5
}
6
7
#[workflow]
8
async fn delivery(ctx: WorkflowContext, order_id: String) -> Result<String> {
9
ctx.set_state("phase", &"packing")?;
10
11
// A query reads state. Its handler takes no arguments, so capture what it
12
// needs; a clone of the context shares the workflow's state.
13
let reader = ctx.clone();
14
ctx.register_query_handler("phase", move || {
15
reader
16
.get_state::<String>("phase")
17
.ok()
18
.flatten()
19
.unwrap_or_default()
20
});
21
22
// An update may change state; its result is journaled.
23
let writer = ctx.clone();
24
ctx.register_update_handler("change_address", move |address: Address| {
25
let writer = writer.clone();
26
async move {
27
writer.set_state("address", &address)?;
28
Ok(true)
29
}
30
});
31
32
// Name the result type, even when it is `()`: `execute_task` is generic
33
// over it.
34
let _: () = ctx.execute_task(pack_order, order_id.clone()).await?;
35
ctx.set_state("phase", &"shipped")?;
36
Ok(order_id)
37
}
38
39
#[task]
40
async fn pack_order(_ctx: TaskContext, order_id: String) -> Result<()> {
41
// Talk to your warehouse system here.
42
let _ = order_id;
43
Ok(())
44
}
  • A query handler is a synchronous Fn() -> R that takes no arguments. Clients call it with handle.query(name, ()). It reads; it should not change state.
  • An update handler is an async Fn(I) -> Future<Output = Result<R>>. Clients call it with handle.update(name, args) and wait for its result, which is journaled.
  • Every workflow also answers the built-in queries workflow_id, workflow_type, run_id, is_replaying, attempt, version, namespace, task_queue and workflow_info (all of these as one JSON object).

Determinism

Deterministic helpers

1
#[derive(Serialize, Deserialize)]
2
pub struct Ticket {
3
pub id: String,
4
pub created_at: i64,
5
pub priority: u8,
6
}
7
8
#[workflow]
9
async fn open_ticket(ctx: WorkflowContext, subject: String) -> Result<Ticket> {
10
// Seeded from the workflow id: a replay draws the same values.
11
let id = format!("T-{}", ctx.rand().uuid());
12
let priority: u8 = ctx.rand().random_range(1..=5);
13
14
// Seconds since the UNIX epoch, read from the journal rather than the
15
// clock: the same at this point on every replay.
16
let created_at = ctx.time().now();
17
18
// Not these: a replay would see different values.
19
// uuid::Uuid::new_v4(), rand::random(), std::time::SystemTime::now()
20
21
let _ = subject;
22
Ok(Ticket {
23
id,
24
created_at,
25
priority,
26
})
27
}
  • ctx.rand() is seeded from the workflow id and advances with each draw: uuid(), random() (a float in [0, 1)), random_range(range), bool(), choose(&items), choose_multiple(&items, n) and shuffle(&mut items).
  • ctx.time().now() is seconds since the UNIX epoch, read from the journal: the workflow's start, then the moment the engine journaled each result, timer or event the workflow has received. It moves forward only as the workflow receives things, and is the same at each point on every replay. started_at(), elapsed(), elapsed_secs() and elapsed_millis() use the same clock.

Logging

The workflow function runs again on every replay, so a log line in it repeats. Log from tasks, or skip logging while replaying:

1
#[workflow]
2
async fn audited(ctx: WorkflowContext, order_id: String) -> Result<String> {
3
// Workflow code runs again on every replay, so a plain log line repeats.
4
if !ctx.is_replaying() {
5
println!("starting {} for {order_id}", ctx.workflow_id());
6
}
7
ctx.execute_task(reserve_stock, order_id).await
8
}

What happens when code changes under a running workflow

Each activation, the worker reports which journaled steps (tasks, timers and child workflows) the code reached. If a run's code skips a step the journal recorded and then issues new work or finishes, or issues different work at a position the journal already has, the activation is held: it is not applied, the run stays open with its journal unchanged, and the worker logs an error that names the workflow and run. The activation is handed back and tried again after 2 seconds, doubling up to 1 minute, until a worker whose code replays the journal picks it up.

Not every change is caught. Reordering steps of different kinds (a task and a timer swapping places) and changing a timer's duration replay without error, against a journal that no longer matches the code.

To change a workflow safely:

  • Let in-flight runs finish before deploying an incompatible change, or run the new code under a new workflow type (#[workflow(name = "process_order_v2")]) beside the old one.
  • Tag each worker with the code release it runs (WorkerBuilder::version_id), which the server records on each execution.
  • ctx.version() is a metadata label only. Don't branch on it.

See Determinism for the concepts.


Restarting with a Fresh History

Every step a workflow takes is added to its journal, and replay reads all of it. A workflow that loops for a long time should restart itself now and then:

1
#[derive(Serialize, Deserialize)]
2
pub struct PollState {
3
pub cursor: u64,
4
}
5
6
#[task]
7
async fn sync_batch(_ctx: TaskContext, cursor: u64) -> Result<u64> {
8
// Copy one batch from the source system and return the next cursor.
9
Ok(cursor + 100)
10
}
11
12
#[workflow]
13
async fn sync_forever(ctx: WorkflowContext, state: PollState) -> Result<()> {
14
let mut cursor = state.cursor;
15
for _ in 0..100 {
16
cursor = ctx.execute_task(sync_batch, cursor).await?;
17
ctx.sleep(std::time::Duration::from_secs(60)).await?;
18
}
19
20
// Start a new run with the same workflow id, an empty history and this
21
// input. Return right after it.
22
ctx.restart_fresh_with(PollState { cursor }, None)
23
}

ctx.restart_fresh() ends the run and starts a new one with the same workflow id, an empty history and no input. restart_fresh_with(input, options) passes an input, and its RestartFreshOptions can change the workflow_type, task_queue or timeout of the new run; changing the type moves a long-running workflow onto new code. Return straight after calling either. A client waiting on the original run's result receives the result of the run that finally completes.


Payload Size

Each payload, a workflow or task input or result, an event or an update, is limited by the engine to 8 MiB by default; the operator can change the limit. A task whose result is too large to send fails with the error type PayloadTooLarge, which is never retried. Every activation also carries the payloads the workflow has received, and a gRPC message is limited to 32 MiB by default (ORCHER_MAX_MESSAGE_BYTES on the worker, and the engine's own setting). Store large data elsewhere and pass a reference (Patterns).


Workflow Metadata

1
#[workflow]
2
async fn describe_self(ctx: WorkflowContext, _input: ()) -> Result<String> {
3
Ok(format!(
4
"{} run {} of type {} in {} on {} (attempt {})",
5
ctx.workflow_id(),
6
ctx.run_id(),
7
ctx.workflow_type(),
8
ctx.namespace(),
9
ctx.task_queue(),
10
ctx.attempt(),
11
))
12
}

ctx.attempt() counts from 1 and increases when a workflow started with a retry policy runs again. ctx.is_replaying() is true while the activation is replaying journaled steps.


WorkflowContext Reference

MethodDescription
execute_task(task, input)Run a task and wait for its result
execute(name, closure)Run an inline step on this worker and journal its result
sleep(duration) / sleep_with_id(id, duration)Durable timer
wait_for_event(name)Wait for the next event with this name
wait_for_event_with_timeout(name, duration)Same, or None after the timeout
execute_child_workflow(type, input)Start a child workflow and wait for its result
start_child_workflow(type, input)Start a child workflow and return a handle
execute_child_workflow_with_options / start_child_workflow_with_optionsThe same with ChildWorkflowOptions
create_session(options)Pin tasks to one worker (Sessions)
set_state(key, &value) / get_state(key)Per-run state, as JSON
register_query_handler(name, handler)Answer a query
register_update_handler(name, handler)Accept an update
rand() / time()Deterministic randomness and time
restart_fresh() / restart_fresh_with(input, options)End the run and start a fresh one
workflow_id(), run_id(), workflow_type(), namespace(), task_queue(), attempt()Metadata
is_replaying()Whether the activation is replaying journaled steps
version()Metadata label; don't branch on it

Next Steps

  • Tasks: retries, non-retryable errors, heartbeats, cancellation
  • Client: starting workflows, events, queries and updates
  • Patterns: sagas, parallel tasks, fan-out/fan-in