Reference

Worker Configuration


A Worker connects to the Orcher server over gRPC, polls one task queue in one namespace for workflow activations and tasks, runs the handlers linked into the binary, and reports their results. It also registers itself with the server, so it shows up in the list of workers, and serves actor operations when actors are defined.


Creating and Running a Worker

1
use orcher_sdk::prelude::*;
2
use std::sync::Arc;
3
4
#[tokio::main(flavor = "current_thread")]
5
async fn main() -> Result<()> {
6
let mut builder = Worker::builder()
7
.server_url("http://localhost:50051")
8
.namespace("payments")
9
.task_queue("orders")
10
.identity("orders-worker-1")
11
.max_concurrent_workflows(100)
12
.max_concurrent_tasks(200);
13
14
// Only when the server requires API keys.
15
if let Ok(key) = std::env::var("ORCHER_API_KEY") {
16
builder = builder.api_key(key);
17
}
18
19
// Label the code release. Without this call, ORCHER_VERSION_ID is used.
20
builder = builder.version_id(env!("CARGO_PKG_VERSION"));
21
22
let worker = Arc::new(builder.build().await?);
23
24
// `run` does not watch for signals itself: ask it to stop from another task.
25
let stopper = Arc::clone(&worker);
26
tokio::spawn(async move {
27
shutdown_signal().await;
28
stopper.shutdown();
29
});
30
31
// Returns once the worker has delivered the results it holds and
32
// deregistered from the server.
33
worker.run().await
34
}
35
36
async fn shutdown_signal() {
37
#[cfg(unix)]
38
{
39
use tokio::signal::unix::{signal, SignalKind};
40
let mut term = signal(SignalKind::terminate()).expect("install SIGTERM handler");
41
tokio::select! {
42
_ = tokio::signal::ctrl_c() => {}
43
_ = term.recv() => {}
44
}
45
}
46
#[cfg(not(unix))]
47
{
48
let _ = tokio::signal::ctrl_c().await;
49
}
50
}

Worker::builder() collects every #[workflow], #[task] and #[actor] linked into the binary. build() checks the configuration and, when actors are defined, registers them with the server; nothing else contacts the server until run().


Configuration

Builder methodDefaultDescription
server_url(url)http://localhost:50051Server gRPC address, with its scheme
namespace(name)defaultThe namespace whose work this worker runs
task_queue(name)defaultThe task queue it polls
api_key(key)NoneSent as authorization: Bearer <key> on every call
organization_id(id)NoneSent as x-organization-id; only for an API key that doesn't belong to an organization itself
tls(ClientTlsConfig)None (follows the URL scheme)A private CA, a client certificate for mutual TLS, or another server name (below)
version_id(id)ORCHER_VERSION_IDThe code release this worker runs (below)
identity(name)orcher-rust-<uuid>Name shown for the worker on the server
max_concurrent_workflows(n)100Workflow activations run at once
max_concurrent_tasks(n)200Tasks run at once
workflow_poller_count(n) / task_poller_count(n)4 / 4Long-poll requests held open at once
actor_poller_count(n) / max_concurrent_actor_operations(n)4 / 100The same for actor operations
max_sessions(n)10Sessions this worker accepts at once
heartbeat_interval(duration)10 sHow often the worker re-announces itself to the server
poll_interval(duration)100 msPause between polls
with_gzip_compression() / with_encryption(&key)NoneDecode payloads the client encoded
interceptor(hook)NoneRun code around every workflow and task execution
1
async fn tuned() -> Result<Worker> {
2
Worker::builder()
3
.server_url("http://localhost:50051")
4
.task_queue("orders")
5
// Long-poll requests held open at once, per kind of work.
6
.workflow_poller_count(8)
7
.task_poller_count(8)
8
// Worker sessions this process accepts at once.
9
.max_sessions(4)
10
// How often the worker re-announces itself to the server.
11
.heartbeat_interval(std::time::Duration::from_secs(10))
12
.build()
13
.await
14
}

The SDK reads two environment variables: ORCHER_VERSION_ID, the default for version_id, and ORCHER_MAX_MESSAGE_BYTES, the largest gRPC message the worker sends or receives (32 MiB by default). Read anything else, such as the server URL or an API key, yourself, as main above does with ORCHER_API_KEY.

Namespaces and authentication

A worker serves exactly one namespace, set with namespace(...); workflows started in other namespaces never reach it. Start workflows in the same namespace, with ClientConfig::with_namespace or StartWorkflowOptions::with_namespace (Client). There is no namespace setting on #[workflow].

When the server requires API keys, give the worker one with api_key(...). It is sent on polls, results, heartbeats and the worker's registration alike.

TLS and mutual TLS

TLS follows the URL scheme. An https:// URL connects over TLS and verifies the server against the system trust store, with no other setting. An http:// URL connects in plaintext, even with TLS settings (the SDK logs a warning), so use https:// whenever the server expects TLS.

tls(...) takes the same ClientTlsConfig as the client and applies it to every connection the worker opens: polls, results, heartbeats, registration and actor calls. Give a CA only for a private or self-signed authority, and a client certificate and key for mutual TLS:

1
use orcher_sdk::client::ClientTlsConfig;
2
use orcher_sdk::prelude::*;
3
4
#[tokio::main(flavor = "current_thread")]
5
async fn main() -> Result<()> {
6
let tls = ClientTlsConfig::new()
7
// Only for a private or self-signed authority; without it the server
8
// is verified against the system trust store.
9
.with_ca_cert_file("/etc/orcher/ca.pem")?
10
// The client certificate and key, for mutual TLS.
11
.with_client_identity_files("/etc/orcher/client.pem", "/etc/orcher/client-key.pem")?;
12
13
let worker = Worker::builder()
14
// TLS needs an https:// URL; an http:// one stays plaintext.
15
.server_url("https://orcher.internal:443")
16
.namespace("payments")
17
.task_queue("orders")
18
.tls(tls)
19
.build()
20
.await?;
21
22
worker.run().await
23
}

build() returns WorkerError::InvalidConfiguration for a client certificate without its key, or PEM that doesn't parse, so bad settings fail at startup rather than on the first poll. When a handshake fails, the error names the cause, such as a certificate the worker doesn't trust or a client certificate the server requires.

TLS and mutual TLS need orcher-sdk 0.7.1 or later. In 0.7.0 and earlier the worker had no TLS setting and couldn't connect to an https:// URL.

Versioning releases

version_id labels the code release a worker runs, for example a git SHA or an image tag. The server records it on each workflow execution the first time a worker claims it. The label is opaque: never parsed or ordered. It defaults to the ORCHER_VERSION_ID environment variable, and leaving both unset changes nothing. This replaces the version attribute on #[workflow], which has no effect.


Graceful Shutdown

worker.run() runs until worker.shutdown() is called or a part of the worker stops. It doesn't watch for signals itself, so share the worker through an Arc and call shutdown() from a signal handler, as main above does. On shutdown the worker:

  1. stops polling for new work;
  2. delivers the results it already holds, waiting up to 10 seconds for this;
  3. deregisters itself from the server, waiting up to 5 seconds, so it stops being listed as connected;
  4. returns from run().

A worker killed without a graceful shutdown doesn't deregister; the server drops its registration once its heartbeats stop. Tasks it was running are retried on another worker once their heartbeats stop (Heartbeats).


Scaling

Run more processes of the same worker, with the same namespace and task queue, to run more work at once. Each workflow activation and each task attempt goes to one of them. Raise max_concurrent_* when a single process has spare CPU and memory, and add processes when it doesn't.

To keep some work apart, for example GPU tasks, run it from workflows started on their own task queue and run a separate worker for that queue. Tasks always run on the queue of the workflow that schedules them.


Health

worker.get_stats() returns counts of executed, succeeded and failed workflows and tasks. worker.get_health() turns them into Healthy, Degraded or Unhealthy from the error rate and the time since the last success, for a health endpoint of your own.


Next Steps

  • Workflows: the workflow context
  • Tasks: retries, heartbeats and cancellation
  • Workers: running workers in operation