Patterns
Workflow shapes that come up again and again, written with the Python SDK. Each one relies on the same guarantee: a step the journal has recorded is not repeated when the workflow resumes.
Compensation (saga)
When a workflow changes several systems and a later step fails, undo the earlier steps. Saga collects a compensation after each step that succeeds and runs them newest first:
1@workflow(name="book-trip")2async def book_trip(ctx: WorkflowContext, hotel_id: str, flight_id: str, amount: int, token: str) -> dict:3 saga = Saga()4 try:5 hotel = await ctx.execute_task(book_hotel, hotel_id=hotel_id)6 saga.add_compensation(lambda: ctx.execute_task(cancel_hotel, reservation_id=hotel["id"]))78 flight = await ctx.execute_task(book_flight, flight_id=flight_id)9 saga.add_compensation(lambda: ctx.execute_task(cancel_flight, reservation_id=flight["id"]))1011 payment = await ctx.execute_task(charge_payment, amount=amount, token=token)12 saga.add_compensation(lambda: ctx.execute_task(refund_payment, payment_id=payment["id"]))13 except Exception:14 # Undo what succeeded, newest first, then fail the workflow.15 await saga.compensate()16 raise1718 saga.commit()19 return {"hotel": hotel["id"], "flight": flight["id"], "payment": payment["id"]}- Compensations are tasks, so they are journaled and retried like any task. A compensation that still fails is logged and the rest still run.
- Register a compensation after its step succeeds, so it can use the step's result (the reservation id here).
saga.add_step(action, compensation)runs an action and registers its compensation in one call when the compensation doesn't need the result. except Exceptionis safe here: the signal the SDK uses to suspend a workflow is not anException, so a workflow that is merely waiting for a task never compensates.
Fan-out and fan-in
To run independent tasks at the same time, schedule them together with asyncio.gather. A loop of awaits would run them one after another:
1@workflow(name="fetch-all")2async def fetch_all(ctx: WorkflowContext, urls: list[str]) -> list[dict]:3 # Every task is scheduled before the workflow waits on any of them.4 results = await asyncio.gather(*(ctx.execute_task(fetch_data, url=url) for url in urls))5 return list(results)For larger units of work, fan out to child workflows. start_child_workflow doesn't wait, so all the children start before the parent waits on any of them:
1@workflow(name="process-regions")2async def process_regions(ctx: WorkflowContext, regions: list[str]) -> dict:3 handles = [4 await ctx.start_child_workflow("process-region", workflow_id=f"region-{r}", args=(r,))5 for r in regions6 ]7 totals = [await handle.result() for handle in handles]8 return {"regions": len(regions), "total": sum(totals)}If one fails, handle.result() raises WorkflowError for it.
Waiting for a person
A workflow can wait days for a decision without holding a worker. Ask, wait for an event with a deadline, and escalate if none comes:
1@workflow(name="expense-approval")2async def expense_approval(ctx: WorkflowContext, request_id: str, approver: str) -> str:3 await ctx.execute_task(send_approval_request, approver=approver, request_id=request_id)45 try:6 decision = await ctx.wait_for_event_with_timeout("decision", timedelta(days=1))7 except TimeoutError:8 await ctx.execute_task(escalate, request_id=request_id)9 return "escalated"1011 return "approved" if decision.get("approved") else "rejected"The approver's tool sends the decision with handle.send_event("decision", {"approved": True}).
State machine
A loop over an explicit state keeps a long, multi-stage process readable, and a query handler makes the current state visible from outside:
1class OrderState(StrEnum):2 CREATED = "created"3 PAYMENT_PENDING = "payment_pending"4 PAID = "paid"5 SHIPPED = "shipped"6 DELIVERED = "delivered"7 CANCELLED = "cancelled"8910@workflow(name="order-lifecycle")11async def order_lifecycle(ctx: WorkflowContext, order_id: str, amount: int, token: str) -> str:12 state = OrderState.CREATED13 ctx.register_query_handler("state", lambda qctx: qctx.get_state("state"))1415 while state not in (OrderState.DELIVERED, OrderState.CANCELLED):16 ctx.set_state("state", state.value)1718 if state == OrderState.CREATED:19 await ctx.execute_task(validate_order, order_id=order_id)20 state = OrderState.PAYMENT_PENDING21 elif state == OrderState.PAYMENT_PENDING:22 payment = await ctx.execute_task(charge_payment, amount=amount, token=token)23 state = OrderState.PAID if payment["success"] else OrderState.CANCELLED24 elif state == OrderState.PAID:25 await ctx.execute_task(ship_order, order_id=order_id)26 state = OrderState.SHIPPED27 elif state == OrderState.SHIPPED:28 await ctx.wait_for_event("delivered")29 state = OrderState.DELIVERED3031 ctx.set_state("state", state.value)32 return state.valueBatches with progress
Process items one task at a time, record progress, and expose it through a query. A failed item is caught as TaskError so the rest of the batch carries on:
1@workflow(name="import-records")2async def import_records(ctx: WorkflowContext, record_ids: list[str]) -> dict:3 progress = {"total": len(record_ids), "done": 0, "failed": 0}4 ctx.register_query_handler("progress", lambda qctx: progress)56 for record_id in record_ids:7 try:8 await ctx.execute_task(process_record, record_id=record_id)9 progress["done"] += 110 except TaskError:11 progress["failed"] += 1 # retries are used up; carry on with the rest1213 return progressOne task per item gives each item its own retries and its own journal entry. For many small items, a task per chunk keeps the journal short. Each task's input and result must stay under the engine's 8 MiB payload limit:
1@workflow(name="import-records-in-chunks")2async def import_records_in_chunks(ctx: WorkflowContext, record_ids: list[str]) -> int:3 imported = 04 # One task per chunk keeps the journal short; keep each input well under 8 MiB.5 for start in range(0, len(record_ids), 500):6 imported += await ctx.execute_task(process_records, record_ids=record_ids[start:start + 500])7 return importedFor a batch with no end, such as a sync that runs forever, process one page per run and restart with fresh history so the journal stays small.
Idempotent side effects
Tasks run at least once: a retried or reassigned attempt may repeat side effects that a previous attempt already made. Pass the services you call an idempotency key that stays the same across attempts; see Tasks run at least once.
Starting a workflow is idempotent through its id: give each piece of work a workflow id derived from it (an order id, a webhook delivery id), and a repeated start is refused rather than run twice. See duplicate starts.