Sessions
A session runs a series of tasks on the same worker. Use one when later tasks depend on something an earlier task left on that worker: a model loaded into memory or onto a GPU, a file downloaded to local disk, a connection held open.
Creating a Session
1use orcher_sdk::prelude::*;2use orcher_sdk::workflow::SessionOptions;3use std::time::Duration;45#[derive(Serialize, Deserialize)]6pub struct InferenceInput {7 pub model: String,8 pub prompts: Vec<String>,9}1011#[task(timeout = 600)]12async fn load_model(_ctx: TaskContext, model: String) -> Result<String> {13 // Load the model into this worker's memory or GPU here.14 Ok(format!("{model} loaded"))15}1617#[task(timeout = 120)]18async fn run_inference(_ctx: TaskContext, prompt: String) -> Result<String> {19 // Use the model this worker loaded.20 Ok(format!("answer to {prompt}"))21}2223#[workflow]24async fn batch_inference(ctx: WorkflowContext, input: InferenceInput) -> Result<Vec<String>> {25 let mut session = ctx26 .create_session(SessionOptions::default().with_creation_timeout(Duration::from_secs(30)))27 .await?;2829 // Every task run through the session goes to the same worker.30 let _loaded: String = session.execute_task(load_model, input.model).await?;31 let mut answers = Vec::new();32 for prompt in input.prompts {33 let answer: String = session.execute_task(run_inference, prompt).await?;34 answers.push(answer);35 }3637 // Release the worker's session slot.38 session.complete();39 Ok(answers)40}ctx.create_session(options) returns a SessionContext. session.execute_task(task, input) takes the same task references and string names as ctx.execute_task, and sends the task to the session's worker. session.complete() releases the worker's session slot; tasks run through the session after that fail.
SessionOptions is in orcher_sdk::workflow, not the prelude.
SessionOptions
| Method | Default | Effect in 0.7 |
|---|---|---|
with_creation_timeout(duration) | 30 s | Limit on the internal task that creates the session |
with_execution_timeout(duration) | 600 s | Sent to the worker, not enforced |
with_max_concurrent_tasks(n) | 1 | Sent to the worker, not enforced |
with_heartbeat_interval(duration) | 5 s | How often the session's worker reports the session alive |
Session Information
1#[workflow]2async fn where_am_i(ctx: WorkflowContext, _input: ()) -> Result<String> {3 let mut session = ctx.create_session(SessionOptions::default()).await?;4 let description = format!(5 "session {} on worker {} via queue {} (open: {})",6 session.session_id(),7 session.worker_identity(),8 session.session_queue(),9 session.is_open(),10 );11 session.complete();12 Ok(description)13}session.info() returns the same values as a SessionInfo, with the session's state.
Worker Capacity
Each worker accepts at most max_sessions sessions at once (10 by default; Worker). Creating a session doesn't wait for a free slot: if the worker that picks up the creation task has none, creation fails at once and create_session returns an error. Size max_sessions for the sessions you expect, and run more workers on the queue if they fill up.
When the Session's Worker Dies
Nothing moves a session to another worker. Tasks sent to the session wait for a worker that is gone; each attempt fails when its timeout passes, and once its retries are used up the workflow receives WorkflowError::TaskFailed. Whatever the worker held, a loaded model or a downloaded file, is gone with it.
Give session tasks timeouts that fit the work, and start over on a new session, a bounded number of times:
1use orcher_sdk::error::WorkflowError;23#[task(timeout = 300, retry = 2)]4async fn transcode(_ctx: TaskContext, path: String) -> Result<String> {5 // Read the file this worker downloaded and transcode it.6 Ok(format!("{path}.mp4"))7}89#[task(timeout = 300)]10async fn download(_ctx: TaskContext, url: String) -> Result<String> {11 // Download to this worker's local disk and return the path.12 Ok(format!("/tmp/{}", url.len()))13}1415#[workflow]16async fn transcode_video(ctx: WorkflowContext, url: String) -> Result<String> {17 let mut attempt = 0;18 loop {19 attempt += 1;20 let mut session = ctx.create_session(SessionOptions::default()).await?;21 let outcome: Result<String> = async {22 let path: String = session.execute_task(download, url.clone()).await?;23 session.execute_task(transcode, path).await24 }25 .await;26 match outcome {27 Ok(output) => {28 session.complete();29 return Ok(output);30 }31 // The worker's local file is gone with the worker: start over on32 // a new session, a bounded number of times.33 Err(Error::Workflow(WorkflowError::TaskFailed { .. })) if attempt < 3 => {34 session.complete();35 }36 // Includes the suspension signal: always pass it on.37 Err(other) => return Err(other),38 }39 }40}How It Works
Sessions are built on task queues; the server has no session concept of its own.
ctx.create_session()schedules an internal__orcher_create_sessiontask on the workflow's task queue.- A worker picks it up, takes a session slot and starts polling a queue of its own, named
{task_queue}__session__{worker_id}. The queue name is the task's result, and is journaled. session.execute_tasksends each task to that queue, which only that worker polls.session.complete()schedules an internal__orcher_complete_sessiontask that releases the slot and stops the polling.
On replay, the session's queue name is read from the journal and the session is not created again.
Limitations
- No migration: a session lives and dies with its worker.
- One worker: everything in a session runs on one worker. For work that can run anywhere, use ordinary tasks.
- Bounded slots: each worker has
max_sessionsslots, and creation fails rather than waits when they are full.