Getting Started

Deploying to Production


Now that you understand how Orcher works, let's deploy it.

Choosing a Deployment Model

Because Orcher workers are just stateless app processes, they run anywhere you already run services — the choice is about your operational comfort, not Orcher's requirements. Three common options, at a glance:

Deployment Models at a Glance
Kubernetes
recommended
ops: you run the cluster
scale: kubectl scale · autoscale
best when: already on K8s; want autoscaling
VMs + LB
full control
ops: you manage the hosts
scale: provision VMs manually
best when: no containers; want SSH-level control
Serverless
zero ops
ops: provider manages all
scale: scale to zero → thousands
best when: bursty load; minimal ops
The database is the only stateful piece in every model — pick workers by the ops model you already know.

Pros:

  • Easy Horizontal Scaling: kubectl scale is all you need
  • Built-in Health Checks: Kubernetes restarts failed pods automatically
  • Service Discovery: Load balancing is free
  • Rolling Updates: Zero-downtime deployments
  • Resource Limits: Prevent runaway processes
  • Mature Ecosystem: Helm charts, operators, ingress controllers

Cons:

  • Kubernetes Learning Curve: If you don't already use K8s, this is a big lift
  • Cluster Overhead: You need to run and maintain a Kubernetes cluster
  • Complexity: K8s adds moving parts (though it manages them for you)

When to Choose This:

  • You're already on Kubernetes
  • You need autoscaling
  • You want a mature deployment platform
  • You have the expertise (or want to build it)

Option 2: VMs with Load Balancer

Pros:

  • Simplicity: No container orchestration needed
  • Full Control: You manage the entire environment
  • Easier Debugging: SSH into machines, inspect directly
  • Lower Overhead: No container runtime or orchestrator

Cons:

  • Manual Scaling: You have to provision VMs yourself
  • Manual Deployments: You coordinate updates across VMs
  • Manual Health Checks: You build monitoring to restart failed processes
  • More Operational Work: Things Kubernetes does for you, you do manually

When to Choose This:

  • You're not on Kubernetes and don't want to learn it
  • You have strict regulatory requirements (air-gapped, on-prem)
  • You prefer full control over automation
  • Your team is stronger with VMs than containers

Option 3: Serverless (Lambda, Cloud Run, Fargate)

Pros:

  • Zero Ops: Provider manages everything
  • Auto-scaling: Scales to zero, scales to thousands
  • Pay Per Use: No idle resource costs
  • Fastest Time to Production: No infra to set up

Cons:

  • Cold Starts: API responses might be slow on first hit
  • Connection Pooling: Requires careful database connection management
  • Vendor Lock-In: Hard to move between clouds
  • Cost at Scale: Can be more expensive than VMs at high utilization

When to Choose This:

  • You want minimal operational overhead
  • Your workload is bursty (not constant high volume)
  • You're okay with cold starts
  • You're comfortable with cloud-specific services

Our Recommendation: Start with Kubernetes if you have the expertise. It's the most mature and flexible. If not, use your cloud's managed container service (ECS, Cloud Run) as a middle ground.

Database Selection

Your database choice is critical because it's your only stateful component.

Cloud Options:

  • AWS RDS for PostgreSQL: Mature, lots of features, automated backups, Multi-AZ HA
  • Google Cloud SQL for PostgreSQL: Similar to RDS, good performance
  • Azure Database for PostgreSQL: Integrated with Azure ecosystem
  • DigitalOcean Managed Databases: Simpler, cheaper, good for smaller deployments

Why Managed Is Good:

  • Automated Backups: Daily backups with point-in-time recovery
  • Automated Patching: Security updates applied automatically
  • High Availability: Multi-AZ with automatic failover (depending on tier)
  • Monitoring: Built-in metrics and logs
  • Scaling: Vertical scaling with a few clicks
  • Less Work: You don't manage the database server

Why Managed Can Be Bad:

  • Cost: usually more expensive than running PostgreSQL yourself
  • Less Control: Can't tune low-level settings
  • Vendor Lock-In: Hard to move between clouds

A starting tier: a 2 vCPU, 4 GB instance with high availability is a reasonable place to start for moderate volume; check your provider's current pricing.

Option 2: Self-Managed PostgreSQL (For Control or Cost)

Why Self-Manage:

  • Cost: 50-70% cheaper than managed
  • Control: Tune every setting
  • Flexibility: Run on your own hardware, in your own datacenter

Why It's Harder:

  • You Manage Everything: Backups, patches, upgrades, monitoring, HA
  • Expertise Required: You need to really know PostgreSQL
  • Time Investment: Ongoing maintenance work

Options:

  • Kubernetes Operator (Zalando Postgres Operator, Crunchy Data, CloudNativePG)
  • VM with Patroni (automated failover)
  • VM with Streaming Replication (manual failover)

When to Self-Manage:

  • You already have PostgreSQL expertise
  • You have strict compliance requirements (data residency, air-gapped)
  • Cost is critical
  • You need very high performance (vertical scaling beyond managed offerings)

Database Sizing

Size the database from a measurement of your own workload rather than a table: the sdk-core benchmark suite runs standard scenarios against any engine, and the engine's metrics show what production load costs it.

Always start smaller and scale up based on actual metrics. It's easier to upgrade than downgrade.

Kubernetes Deployment (Step by Step)

Let's walk through a complete production deployment on Kubernetes.

Prerequisites

Before you begin:

1
# 1. Kubernetes cluster (1.24+)
2
kubectl version
3
4
# 2. Helm (for easier deployment)
5
helm version
6
7
# 3. kubectl context set to your cluster
8
kubectl config current-context
9
10
# 4. Container registry access
11
docker login your-registry.com

Step 1: Create Namespace and Secrets

1
# Create a dedicated namespace
2
kubectl create namespace orcher
3
4
# Create database credentials secret
5
kubectl create secret generic postgres-credentials \
6
--from-literal=username=orcher \
7
--from-literal=password='your-super-secure-password-here' \
8
--from-literal=database=orcher \
9
-n orcher
10
11
# Create connection string secret (for app)
12
kubectl create secret generic orcher-config \
13
--from-literal=database-url='postgresql://orcher:your-super-secure-password-here@postgres:5432/orcher' \
14
-n orcher

Security Note: In production, use a secrets manager (AWS Secrets Manager, HashiCorp Vault, Kubernetes External Secrets) instead of kubectl secrets.

Step 2: Deploy PostgreSQL

For production, use a managed service (RDS, Cloud SQL). For self-hosting, use this StatefulSet:

1
# postgres.yaml
2
apiVersion: v1
3
kind: Service
4
metadata:
5
name: postgres
6
namespace: orcher
7
spec:
8
ports:
9
- port: 5432
10
targetPort: 5432
11
selector:
12
app: postgres
13
clusterIP: None # Headless service for StatefulSet
14
---
15
apiVersion: apps/v1
16
kind: StatefulSet
17
metadata:
18
name: postgres
19
namespace: orcher
20
spec:
21
serviceName: postgres
22
replicas: 1
23
selector:
24
matchLabels:
25
app: postgres
26
template:
27
metadata:
28
labels:
29
app: postgres
30
spec:
31
containers:
32
- name: postgres
33
image: postgres:15
34
ports:
35
- containerPort: 5432
36
name: postgres
37
env:
38
- name: POSTGRES_DB
39
valueFrom:
40
secretKeyRef:
41
name: postgres-credentials
42
key: database
43
- name: POSTGRES_USER
44
valueFrom:
45
secretKeyRef:
46
name: postgres-credentials
47
key: username
48
- name: POSTGRES_PASSWORD
49
valueFrom:
50
secretKeyRef:
51
name: postgres-credentials
52
key: password
53
- name: PGDATA
54
value: /var/lib/postgresql/data/pgdata
55
volumeMounts:
56
- name: postgres-storage
57
mountPath: /var/lib/postgresql/data
58
resources:
59
requests:
60
memory: "2Gi"
61
cpu: "1"
62
limits:
63
memory: "4Gi"
64
cpu: "2"
65
livenessProbe:
66
exec:
67
command:
68
- pg_isready
69
- -U
70
- orcher
71
initialDelaySeconds: 30
72
periodSeconds: 10
73
readinessProbe:
74
exec:
75
command:
76
- pg_isready
77
- -U
78
- orcher
79
initialDelaySeconds: 5
80
periodSeconds: 5
81
volumeClaimTemplates:
82
- metadata:
83
name: postgres-storage
84
spec:
85
accessModes: [ "ReadWriteOnce" ]
86
storageClassName: "fast-ssd" # Use your cluster's fast storage class
87
resources:
88
requests:
89
storage: 100Gi

Apply it:

1
kubectl apply -f postgres.yaml
2
3
# Wait for it to be ready
4
kubectl wait --for=condition=ready pod -l app=postgres -n orcher --timeout=300s
5
6
# Verify
7
kubectl get pods -n orcher
8
# Should show postgres-0 Running

Initialize the database schema:

1
# Connect to the database
2
kubectl exec -it postgres-0 -n orcher -- psql -U orcher
3
4
# Run initialization SQL
5
CREATE TABLE IF NOT EXISTS workflow_queue (
6
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
7
workflow_name TEXT NOT NULL,
8
input JSONB,
9
status TEXT NOT NULL DEFAULT 'pending',
10
retry_count INT DEFAULT 0,
11
next_retry_at TIMESTAMP,
12
created_at TIMESTAMP DEFAULT NOW(),
13
updated_at TIMESTAMP DEFAULT NOW(),
14
completed_at TIMESTAMP
15
);
16
17
CREATE INDEX idx_workflow_queue_status ON workflow_queue(status, created_at);
18
CREATE INDEX idx_workflow_queue_retry ON workflow_queue(next_retry_at) WHERE status = 'pending';
19
20
CREATE TABLE IF NOT EXISTS workflow_execution_journal (
21
workflow_id UUID NOT NULL,
22
step_id INT NOT NULL,
23
task_name TEXT NOT NULL,
24
input JSONB,
25
output JSONB,
26
status TEXT NOT NULL,
27
error TEXT,
28
created_at TIMESTAMP DEFAULT NOW(),
29
PRIMARY KEY (workflow_id, step_id)
30
);
31
32
CREATE INDEX idx_workflow_execution_journal_lookup ON workflow_execution_journal(workflow_id, step_id);
33
CREATE INDEX idx_workflow_execution_journal_workflow ON workflow_execution_journal(workflow_id);
34
35
-- Exit
36
\q

Step 3: Build and Push Your Application Image

1
# Dockerfile
2
# Multi-stage build for small final image
3
FROM rust:1.75 as builder
4
5
WORKDIR /app
6
7
# Copy dependency manifests
8
COPY Cargo.toml Cargo.lock ./
9
10
# Create dummy main to cache dependencies
11
RUN mkdir src && \
12
echo "fn main() {}" > src/main.rs && \
13
cargo build --release && \
14
rm -rf src
15
16
# Copy real source code
17
COPY src ./src
18
19
# Build real application
20
RUN touch src/main.rs && cargo build --release
21
22
# Final stage - minimal runtime image
23
FROM debian:bookworm-slim
24
25
# Install runtime dependencies
26
RUN apt-get update && apt-get install -y \
27
ca-certificates \
28
libssl3 \
29
libpq5 \
30
&& rm -rf /var/lib/apt/lists/*
31
32
# Copy binary from builder
33
COPY --from=builder /app/target/release/orcher-app /usr/local/bin/orcher-app
34
35
# Create non-root user
36
RUN useradd -m -u 1000 orcher
37
38
USER orcher
39
40
EXPOSE 3000
41
42
# Health check endpoint
43
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
44
CMD curl -f http://localhost:3000/health || exit 1
45
46
CMD ["orcher-app"]

Build and push:

1
# Build
2
docker build -t your-registry.com/orcher-app:v1.0 .
3
4
# Push
5
docker push your-registry.com/orcher-app:v1.0
6
7
# Tag as latest
8
docker tag your-registry.com/orcher-app:v1.0 your-registry.com/orcher-app:latest
9
docker push your-registry.com/orcher-app:latest

Step 4: Deploy Application (API + Workers)

1
# orcher-app.yaml
2
---
3
# ConfigMap for shared configuration
4
apiVersion: v1
5
kind: ConfigMap
6
metadata:
7
name: orcher-config
8
namespace: orcher
9
data:
10
LOG_LEVEL: "info"
11
WORKER_CONCURRENCY: "10"
12
POLL_INTERVAL_MS: "100"
13
MAX_WORKFLOW_DURATION_SECS: "3600"
14
---
15
# API Deployment
16
apiVersion: apps/v1
17
kind: Deployment
18
metadata:
19
name: orcher-api
20
namespace: orcher
21
labels:
22
app: orcher-api
23
component: api
24
spec:
25
replicas: 3
26
strategy:
27
type: RollingUpdate
28
rollingUpdate:
29
maxSurge: 1
30
maxUnavailable: 0 # Zero downtime
31
selector:
32
matchLabels:
33
app: orcher-api
34
template:
35
metadata:
36
labels:
37
app: orcher-api
38
component: api
39
spec:
40
# Spread across availability zones
41
affinity:
42
podAntiAffinity:
43
preferredDuringSchedulingIgnoredDuringExecution:
44
- weight: 100
45
podAffinityTerm:
46
labelSelector:
47
matchExpressions:
48
- key: app
49
operator: In
50
values:
51
- orcher-api
52
topologyKey: topology.kubernetes.io/zone
53
containers:
54
- name: api
55
image: your-registry.com/orcher-app:latest
56
ports:
57
- containerPort: 3000
58
name: http
59
protocol: TCP
60
env:
61
- name: MODE
62
value: "api"
63
- name: DATABASE_URL
64
valueFrom:
65
secretKeyRef:
66
name: orcher-config
67
key: database-url
68
envFrom:
69
- configMapRef:
70
name: orcher-config
71
resources:
72
requests:
73
memory: "512Mi"
74
cpu: "500m"
75
limits:
76
memory: "1Gi"
77
cpu: "1000m"
78
livenessProbe:
79
httpGet:
80
path: /health
81
port: 3000
82
initialDelaySeconds: 10
83
periodSeconds: 10
84
timeoutSeconds: 5
85
failureThreshold: 3
86
readinessProbe:
87
httpGet:
88
path: /ready
89
port: 3000
90
initialDelaySeconds: 5
91
periodSeconds: 5
92
timeoutSeconds: 3
93
failureThreshold: 2
94
---
95
# Worker Deployment
96
apiVersion: apps/v1
97
kind: Deployment
98
metadata:
99
name: orcher-worker
100
namespace: orcher
101
labels:
102
app: orcher-worker
103
component: worker
104
spec:
105
replicas: 5
106
strategy:
107
type: RollingUpdate
108
rollingUpdate:
109
maxSurge: 2
110
maxUnavailable: 1
111
selector:
112
matchLabels:
113
app: orcher-worker
114
template:
115
metadata:
116
labels:
117
app: orcher-worker
118
component: worker
119
spec:
120
affinity:
121
podAntiAffinity:
122
preferredDuringSchedulingIgnoredDuringExecution:
123
- weight: 100
124
podAffinityTerm:
125
labelSelector:
126
matchExpressions:
127
- key: app
128
operator: In
129
values:
130
- orcher-worker
131
topologyKey: topology.kubernetes.io/zone
132
containers:
133
- name: worker
134
image: your-registry.com/orcher-app:latest
135
env:
136
- name: MODE
137
value: "worker"
138
- name: DATABASE_URL
139
valueFrom:
140
secretKeyRef:
141
name: orcher-config
142
key: database-url
143
envFrom:
144
- configMapRef:
145
name: orcher-config
146
resources:
147
requests:
148
memory: "1Gi"
149
cpu: "1000m"
150
limits:
151
memory: "2Gi"
152
cpu: "2000m"
153
livenessProbe:
154
exec:
155
command:
156
- /bin/sh
157
- -c
158
- "pgrep -f worker || exit 1"
159
initialDelaySeconds: 30
160
periodSeconds: 30
161
timeoutSeconds: 5
162
failureThreshold: 3
163
---
164
# Service for API
165
apiVersion: v1
166
kind: Service
167
metadata:
168
name: orcher-api
169
namespace: orcher
170
spec:
171
type: LoadBalancer
172
ports:
173
- port: 80
174
targetPort: 3000
175
protocol: TCP
176
name: http
177
selector:
178
app: orcher-api
179
---
180
# Horizontal Pod Autoscaler for Workers
181
apiVersion: autoscaling/v2
182
kind: HorizontalPodAutoscaler
183
metadata:
184
name: orcher-worker-hpa
185
namespace: orcher
186
spec:
187
scaleTargetRef:
188
apiVersion: apps/v1
189
kind: Deployment
190
name: orcher-worker
191
minReplicas: 3
192
maxReplicas: 20
193
metrics:
194
- type: Resource
195
resource:
196
name: cpu
197
target:
198
type: Utilization
199
averageUtilization: 70
200
- type: Resource
201
resource:
202
name: memory
203
target:
204
type: Utilization
205
averageUtilization: 80
206
behavior:
207
scaleDown:
208
stabilizationWindowSeconds: 300 # Wait 5 min before scaling down
209
policies:
210
- type: Percent
211
value: 50
212
periodSeconds: 60
213
scaleUp:
214
stabilizationWindowSeconds: 0 # Scale up immediately
215
policies:
216
- type: Percent
217
value: 100
218
periodSeconds: 30

Deploy everything:

1
# Apply the manifests
2
kubectl apply -f orcher-app.yaml
3
4
# Watch the rollout
5
kubectl rollout status deployment/orcher-api -n orcher
6
kubectl rollout status deployment/orcher-worker -n orcher
7
8
# Get the LoadBalancer IP
9
kubectl get svc orcher-api -n orcher
10
11
# Test the API
12
export API_URL=$(kubectl get svc orcher-api -n orcher -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
13
curl http://$API_URL/health
14
# Should return: {"status": "healthy"}

Verification checklist:

1
# Check all pods are running
2
kubectl get pods -n orcher
3
# Should show:
4
# - postgres-0 (Running)
5
# - orcher-api-xxxxx (3 replicas, all Running)
6
# - orcher-worker-xxxxx (5 replicas, all Running)
7
8
# Check logs for errors
9
kubectl logs -l app=orcher-api -n orcher --tail=100
10
kubectl logs -l app=orcher-worker -n orcher --tail=100
11
12
# Check database connectivity from app
13
kubectl exec -it deployment/orcher-api -n orcher -- /bin/sh
14
# Inside pod:
15
# Test database connection (if you have psql)

Congratulations! Orcher is now running in production.


Next Steps