Sessions
Tasks normally run on whichever worker polls their queue first. A session sends a series of tasks to one worker, so they can share what that worker holds locally:
- files a task downloaded to the worker's disk, for the next task to transform;
- a model loaded into memory, for several inference tasks to use; or
- open connections or other in-process state.
Creating a session
Define the tasks as usual:
1@task(name="download", timeout=600.0)2async def download(ctx: TaskContext, url: str) -> str:3 # Fetch the file to this worker's disk here.4 path = WORK_DIR / Path(url).name5 return str(path)678@task(name="transcode", timeout=1800.0)9async def transcode(ctx: TaskContext, path: str) -> str:10 # Reads the file download left on this same machine.11 return path + ".mp4"121314@task(name="upload", timeout=600.0)15async def upload(ctx: TaskContext, path: str) -> str:16 return f"https://cdn.example.com/{Path(path).name}"Then create a session in the workflow and run the tasks through it. session.execute_task() takes the same arguments as ctx.execute_task(); the only difference is where the task runs:
1@workflow(name="transcode-video")2async def transcode_video(ctx: WorkflowContext, url: str) -> str:3 session = await ctx.create_session(4 SessionOptions(creation_timeout=timedelta(seconds=30))5 )67 try:8 local = await session.execute_task(download, url=url)9 output = await session.execute_task(transcode, path=local)10 link: str = await session.execute_task(upload, path=output)11 except Exception:12 # Not `finally`: that would also run each time the workflow suspends.13 session.complete()14 raise1516 session.complete() # frees the worker's session slot17 return linkSessionOptions is imported from orcher.workflow. session.complete() releases the worker's session slot. Call it on every path out of the session, including failures, but not from a finally block: finally also runs each time the workflow suspends, and would close the session after its first task.
When no worker has room
Each worker hosts up to 10 sessions at a time. A worker with no free slot doesn't wait for one: the session's creation fails, the engine retries it like any task (possibly on another worker), and once those attempts are used up create_session() raises TaskError. Handle that by waiting and trying again. The same loop starts over in a new session when a task in the session fails, which is also how to recover from losing the session's worker (see Limitations):
1@workflow(name="transcode-video-retrying")2async def transcode_video_retrying(ctx: WorkflowContext, url: str) -> str:3 for _ in range(3):4 try:5 session = await ctx.create_session()6 except TaskError:7 # No worker had a free session slot. Wait, then ask again.8 await ctx.sleep(timedelta(seconds=30))9 continue10 # If the session's worker stops, its queue is never polled again:11 # queue_timeout makes a stranded task fail instead of waiting forever.12 wait = timedelta(minutes=5)13 try:14 local = await session.execute_task(download, url=url, queue_timeout=wait)15 output = await session.execute_task(transcode, path=local, queue_timeout=wait)16 link: str = await session.execute_task(upload, path=output, queue_timeout=wait)17 except TaskError:18 session.complete()19 continue # start over in a new session20 session.complete()21 return link22 raise RuntimeError("could not finish in a session")SessionOptions
| Field | Default | Description |
|---|---|---|
creation_timeout | timedelta(seconds=30) | How long creating the session may take; it is the timeout of the creation step |
execution_timeout | timedelta(seconds=600) | How long the session may stay open; past it the worker closes the session and frees its slot |
max_concurrent_tasks | 1 | Sent to the worker, but not enforced |
heartbeat_interval | timedelta(seconds=5) | Sent to the worker, but not used |
Because execution_timeout is not enforced, a session that is never completed holds its slot until the worker restarts.
Session details
1@workflow(name="session-info")2async def session_info(ctx: WorkflowContext) -> dict:3 session = await ctx.create_session()4 details = {5 "session_id": session.session_id,6 "worker": session.worker_identity, # an id the worker generates7 "queue": session.session_queue,8 "open": session.is_open,9 }10 session.complete()11 return detailsHow it works
Sessions are built from task queues; the engine has no separate session concept.
create_session()schedules an internal task,__orcher_create_session, on the workflow's queue.- The worker that picks it up takes a session slot and starts polling a queue of its own, named
<queue>__session__<id>, where the id is generated by that worker process. - The task's result, the queue name, is journaled, so a replay reuses the same session.
session.execute_task()schedules each task on that queue, which only that worker polls.session.complete()schedules__orcher_complete_sessionon the session queue; the worker stops polling it and frees the slot.
Limitations
- A session ends with its worker. If the worker stops while one of the session's tasks is running, that task times out (its heartbeat timeout, a minute by default) and the engine fails it and every other task on the session's queue as lost; the workflow gets a
TaskErrorand can open a new session. A task only waiting on the queue is caught by its ownqueue_timeout, which by default is none, so give session tasks aqueue_timeout, catch theTaskError, and start over in a new session, as the example above does. - Everything runs on one machine. For work that can run anywhere, use ordinary tasks, which spread across workers.
- Capacity is fixed. Ten sessions per worker, not configurable from Python; run more workers for more sessions.