Reference

Sessions


A session runs a series of tasks on the same worker. Use one when later tasks depend on something an earlier task left on that worker: a model loaded into memory or onto a GPU, a file downloaded to local disk, a connection held open.


Creating a Session

1
use orcher_sdk::prelude::*;
2
use orcher_sdk::workflow::SessionOptions;
3
use std::time::Duration;
4
5
#[derive(Serialize, Deserialize)]
6
pub struct InferenceInput {
7
pub model: String,
8
pub prompts: Vec<String>,
9
}
10
11
#[task(timeout = 600)]
12
async fn load_model(_ctx: TaskContext, model: String) -> Result<String> {
13
// Load the model into this worker's memory or GPU here.
14
Ok(format!("{model} loaded"))
15
}
16
17
#[task(timeout = 120)]
18
async fn run_inference(_ctx: TaskContext, prompt: String) -> Result<String> {
19
// Use the model this worker loaded.
20
Ok(format!("answer to {prompt}"))
21
}
22
23
#[workflow]
24
async fn batch_inference(ctx: WorkflowContext, input: InferenceInput) -> Result<Vec<String>> {
25
let mut session = ctx
26
.create_session(SessionOptions::default().with_creation_timeout(Duration::from_secs(30)))
27
.await?;
28
29
// Every task run through the session goes to the same worker.
30
let _loaded: String = session.execute_task(load_model, input.model).await?;
31
let mut answers = Vec::new();
32
for prompt in input.prompts {
33
let answer: String = session.execute_task(run_inference, prompt).await?;
34
answers.push(answer);
35
}
36
37
// Release the worker's session slot.
38
session.complete();
39
Ok(answers)
40
}

ctx.create_session(options) returns a SessionContext. session.execute_task(task, input) takes the same task references and string names as ctx.execute_task, and sends the task to the session's worker. session.complete() releases the worker's session slot; tasks run through the session after that fail.

SessionOptions is in orcher_sdk::workflow, not the prelude.


SessionOptions

MethodDefaultEffect in 0.7
with_creation_timeout(duration)30 sLimit on the internal task that creates the session
with_execution_timeout(duration)600 sSent to the worker, not enforced
with_max_concurrent_tasks(n)1Sent to the worker, not enforced
with_heartbeat_interval(duration)5 sHow often the session's worker reports the session alive

Session Information

1
#[workflow]
2
async fn where_am_i(ctx: WorkflowContext, _input: ()) -> Result<String> {
3
let mut session = ctx.create_session(SessionOptions::default()).await?;
4
let description = format!(
5
"session {} on worker {} via queue {} (open: {})",
6
session.session_id(),
7
session.worker_identity(),
8
session.session_queue(),
9
session.is_open(),
10
);
11
session.complete();
12
Ok(description)
13
}

session.info() returns the same values as a SessionInfo, with the session's state.


Worker Capacity

Each worker accepts at most max_sessions sessions at once (10 by default; Worker). Creating a session doesn't wait for a free slot: if the worker that picks up the creation task has none, creation fails at once and create_session returns an error. Size max_sessions for the sessions you expect, and run more workers on the queue if they fill up.


When the Session's Worker Dies

Nothing moves a session to another worker. Tasks sent to the session wait for a worker that is gone; each attempt fails when its timeout passes, and once its retries are used up the workflow receives WorkflowError::TaskFailed. Whatever the worker held, a loaded model or a downloaded file, is gone with it.

Give session tasks timeouts that fit the work, and start over on a new session, a bounded number of times:

1
use orcher_sdk::error::WorkflowError;
2
3
#[task(timeout = 300, retry = 2)]
4
async fn transcode(_ctx: TaskContext, path: String) -> Result<String> {
5
// Read the file this worker downloaded and transcode it.
6
Ok(format!("{path}.mp4"))
7
}
8
9
#[task(timeout = 300)]
10
async fn download(_ctx: TaskContext, url: String) -> Result<String> {
11
// Download to this worker's local disk and return the path.
12
Ok(format!("/tmp/{}", url.len()))
13
}
14
15
#[workflow]
16
async fn transcode_video(ctx: WorkflowContext, url: String) -> Result<String> {
17
let mut attempt = 0;
18
loop {
19
attempt += 1;
20
let mut session = ctx.create_session(SessionOptions::default()).await?;
21
let outcome: Result<String> = async {
22
let path: String = session.execute_task(download, url.clone()).await?;
23
session.execute_task(transcode, path).await
24
}
25
.await;
26
match outcome {
27
Ok(output) => {
28
session.complete();
29
return Ok(output);
30
}
31
// The worker's local file is gone with the worker: start over on
32
// a new session, a bounded number of times.
33
Err(Error::Workflow(WorkflowError::TaskFailed { .. })) if attempt < 3 => {
34
session.complete();
35
}
36
// Includes the suspension signal: always pass it on.
37
Err(other) => return Err(other),
38
}
39
}
40
}

How It Works

Sessions are built on task queues; the server has no session concept of its own.

  1. ctx.create_session() schedules an internal __orcher_create_session task on the workflow's task queue.
  2. A worker picks it up, takes a session slot and starts polling a queue of its own, named {task_queue}__session__{worker_id}. The queue name is the task's result, and is journaled.
  3. session.execute_task sends each task to that queue, which only that worker polls.
  4. session.complete() schedules an internal __orcher_complete_session task that releases the slot and stops the polling.

On replay, the session's queue name is read from the journal and the session is not created again.


Limitations

  • No migration: a session lives and dies with its worker.
  • One worker: everything in a session runs on one worker. For work that can run anywhere, use ordinary tasks.
  • Bounded slots: each worker has max_sessions slots, and creation fails rather than waits when they are full.

Next Steps

  • Tasks: timeouts and retries for session tasks
  • Worker: max_sessions and scaling
  • Workflows: handling task failures