Reference

Sessions


Tasks normally run on whichever worker polls their queue first. A session sends a series of tasks to one worker, so they can share what that worker holds locally:

  • files a task downloaded to the worker's disk, for the next task to transform;
  • a model loaded into memory, for several inference tasks to use; or
  • open connections or other in-process state.

Creating a session

Define the tasks as usual:

1
@task(name="download", timeout=600.0)
2
async def download(ctx: TaskContext, url: str) -> str:
3
# Fetch the file to this worker's disk here.
4
path = WORK_DIR / Path(url).name
5
return str(path)
6
7
8
@task(name="transcode", timeout=1800.0)
9
async def transcode(ctx: TaskContext, path: str) -> str:
10
# Reads the file download left on this same machine.
11
return path + ".mp4"
12
13
14
@task(name="upload", timeout=600.0)
15
async def upload(ctx: TaskContext, path: str) -> str:
16
return f"https://cdn.example.com/{Path(path).name}"

Then create a session in the workflow and run the tasks through it. session.execute_task() takes the same arguments as ctx.execute_task(); the only difference is where the task runs:

1
@workflow(name="transcode-video")
2
async def transcode_video(ctx: WorkflowContext, url: str) -> str:
3
session = await ctx.create_session(
4
SessionOptions(creation_timeout=timedelta(seconds=30))
5
)
6
7
try:
8
local = await session.execute_task(download, url=url)
9
output = await session.execute_task(transcode, path=local)
10
link: str = await session.execute_task(upload, path=output)
11
except Exception:
12
# Not `finally`: that would also run each time the workflow suspends.
13
session.complete()
14
raise
15
16
session.complete() # frees the worker's session slot
17
return link

SessionOptions is imported from orcher.workflow. session.complete() releases the worker's session slot. Call it on every path out of the session, including failures, but not from a finally block: finally also runs each time the workflow suspends, and would close the session after its first task.


When no worker has room

Each worker hosts up to 10 sessions at a time. A worker with no free slot doesn't wait for one: the session's creation fails, the engine retries it like any task (possibly on another worker), and once those attempts are used up create_session() raises TaskError. Handle that by waiting and trying again. The same loop starts over in a new session when a task in the session fails, which is also how to recover from losing the session's worker (see Limitations):

1
@workflow(name="transcode-video-retrying")
2
async def transcode_video_retrying(ctx: WorkflowContext, url: str) -> str:
3
for _ in range(3):
4
try:
5
session = await ctx.create_session()
6
except TaskError:
7
# No worker had a free session slot. Wait, then ask again.
8
await ctx.sleep(timedelta(seconds=30))
9
continue
10
# If the session's worker stops, its queue is never polled again:
11
# queue_timeout makes a stranded task fail instead of waiting forever.
12
wait = timedelta(minutes=5)
13
try:
14
local = await session.execute_task(download, url=url, queue_timeout=wait)
15
output = await session.execute_task(transcode, path=local, queue_timeout=wait)
16
link: str = await session.execute_task(upload, path=output, queue_timeout=wait)
17
except TaskError:
18
session.complete()
19
continue # start over in a new session
20
session.complete()
21
return link
22
raise RuntimeError("could not finish in a session")

SessionOptions

FieldDefaultDescription
creation_timeouttimedelta(seconds=30)How long creating the session may take; it is the timeout of the creation step
execution_timeouttimedelta(seconds=600)How long the session may stay open; past it the worker closes the session and frees its slot
max_concurrent_tasks1Sent to the worker, but not enforced
heartbeat_intervaltimedelta(seconds=5)Sent to the worker, but not used

Because execution_timeout is not enforced, a session that is never completed holds its slot until the worker restarts.


Session details

1
@workflow(name="session-info")
2
async def session_info(ctx: WorkflowContext) -> dict:
3
session = await ctx.create_session()
4
details = {
5
"session_id": session.session_id,
6
"worker": session.worker_identity, # an id the worker generates
7
"queue": session.session_queue,
8
"open": session.is_open,
9
}
10
session.complete()
11
return details

How it works

Sessions are built from task queues; the engine has no separate session concept.

  1. create_session() schedules an internal task, __orcher_create_session, on the workflow's queue.
  2. The worker that picks it up takes a session slot and starts polling a queue of its own, named <queue>__session__<id>, where the id is generated by that worker process.
  3. The task's result, the queue name, is journaled, so a replay reuses the same session.
  4. session.execute_task() schedules each task on that queue, which only that worker polls.
  5. session.complete() schedules __orcher_complete_session on the session queue; the worker stops polling it and frees the slot.

Limitations

  • A session ends with its worker. If the worker stops while one of the session's tasks is running, that task times out (its heartbeat timeout, a minute by default) and the engine fails it and every other task on the session's queue as lost; the workflow gets a TaskError and can open a new session. A task only waiting on the queue is caught by its own queue_timeout, which by default is none, so give session tasks a queue_timeout, catch the TaskError, and start over in a new session, as the example above does.
  • Everything runs on one machine. For work that can run anywhere, use ordinary tasks, which spread across workers.
  • Capacity is fixed. Ten sessions per worker, not configurable from Python; run more workers for more sessions.

Next steps

  • Tasks: timeouts, retries and heartbeats
  • Workflows: handling task failures
  • Worker: running more workers