Deploying to Production
Now that you understand how Orcher works, let's deploy it.
Choosing a Deployment Model
Because Orcher workers are just stateless app processes, they run anywhere you already run services — the choice is about your operational comfort, not Orcher's requirements. Three common options, at a glance:
Option 1: Kubernetes (Recommended)
Pros:
- Easy Horizontal Scaling:
kubectl scaleis all you need - Built-in Health Checks: Kubernetes restarts failed pods automatically
- Service Discovery: Load balancing is free
- Rolling Updates: Zero-downtime deployments
- Resource Limits: Prevent runaway processes
- Mature Ecosystem: Helm charts, operators, ingress controllers
Cons:
- Kubernetes Learning Curve: If you don't already use K8s, this is a big lift
- Cluster Overhead: You need to run and maintain a Kubernetes cluster
- Complexity: K8s adds moving parts (though it manages them for you)
When to Choose This:
- You're already on Kubernetes
- You need autoscaling
- You want a mature deployment platform
- You have the expertise (or want to build it)
Option 2: VMs with Load Balancer
Pros:
- Simplicity: No container orchestration needed
- Full Control: You manage the entire environment
- Easier Debugging: SSH into machines, inspect directly
- Lower Overhead: No container runtime or orchestrator
Cons:
- Manual Scaling: You have to provision VMs yourself
- Manual Deployments: You coordinate updates across VMs
- Manual Health Checks: You build monitoring to restart failed processes
- More Operational Work: Things Kubernetes does for you, you do manually
When to Choose This:
- You're not on Kubernetes and don't want to learn it
- You have strict regulatory requirements (air-gapped, on-prem)
- You prefer full control over automation
- Your team is stronger with VMs than containers
Option 3: Serverless (Lambda, Cloud Run, Fargate)
Pros:
- Zero Ops: Provider manages everything
- Auto-scaling: Scales to zero, scales to thousands
- Pay Per Use: No idle resource costs
- Fastest Time to Production: No infra to set up
Cons:
- Cold Starts: API responses might be slow on first hit
- Connection Pooling: Requires careful database connection management
- Vendor Lock-In: Hard to move between clouds
- Cost at Scale: Can be more expensive than VMs at high utilization
When to Choose This:
- You want minimal operational overhead
- Your workload is bursty (not constant high volume)
- You're okay with cold starts
- You're comfortable with cloud-specific services
Our Recommendation: Start with Kubernetes if you have the expertise. It's the most mature and flexible. If not, use your cloud's managed container service (ECS, Cloud Run) as a middle ground.
Database Selection
Your database choice is critical because it's your only stateful component.
Option 1: Managed PostgreSQL (Recommended for Most)
Cloud Options:
- AWS RDS for PostgreSQL: Mature, lots of features, automated backups, Multi-AZ HA
- Google Cloud SQL for PostgreSQL: Similar to RDS, good performance
- Azure Database for PostgreSQL: Integrated with Azure ecosystem
- DigitalOcean Managed Databases: Simpler, cheaper, good for smaller deployments
Why Managed Is Good:
- Automated Backups: Daily backups with point-in-time recovery
- Automated Patching: Security updates applied automatically
- High Availability: Multi-AZ with automatic failover (depending on tier)
- Monitoring: Built-in metrics and logs
- Scaling: Vertical scaling with a few clicks
- Less Work: You don't manage the database server
Why Managed Can Be Bad:
- Cost: usually more expensive than running PostgreSQL yourself
- Less Control: Can't tune low-level settings
- Vendor Lock-In: Hard to move between clouds
A starting tier: a 2 vCPU, 4 GB instance with high availability is a reasonable place to start for moderate volume; check your provider's current pricing.
Option 2: Self-Managed PostgreSQL (For Control or Cost)
Why Self-Manage:
- Cost: 50-70% cheaper than managed
- Control: Tune every setting
- Flexibility: Run on your own hardware, in your own datacenter
Why It's Harder:
- You Manage Everything: Backups, patches, upgrades, monitoring, HA
- Expertise Required: You need to really know PostgreSQL
- Time Investment: Ongoing maintenance work
Options:
- Kubernetes Operator (Zalando Postgres Operator, Crunchy Data, CloudNativePG)
- VM with Patroni (automated failover)
- VM with Streaming Replication (manual failover)
When to Self-Manage:
- You already have PostgreSQL expertise
- You have strict compliance requirements (data residency, air-gapped)
- Cost is critical
- You need very high performance (vertical scaling beyond managed offerings)
Database Sizing
Size the database from a measurement of your own workload rather than a table: the sdk-core benchmark suite runs standard scenarios against any engine, and the engine's metrics show what production load costs it.
Always start smaller and scale up based on actual metrics. It's easier to upgrade than downgrade.
Kubernetes Deployment (Step by Step)
Let's walk through a complete production deployment on Kubernetes.
Prerequisites
Before you begin:
1# 1. Kubernetes cluster (1.24+)2kubectl version34# 2. Helm (for easier deployment)5helm version67# 3. kubectl context set to your cluster8kubectl config current-context910# 4. Container registry access11docker login your-registry.comStep 1: Create Namespace and Secrets
1# Create a dedicated namespace2kubectl create namespace orcher34# Create database credentials secret5kubectl create secret generic postgres-credentials \6 --from-literal=username=orcher \7 --from-literal=password='your-super-secure-password-here' \8 --from-literal=database=orcher \9 -n orcher1011# Create connection string secret (for app)12kubectl create secret generic orcher-config \13 --from-literal=database-url='postgresql://orcher:your-super-secure-password-here@postgres:5432/orcher' \14 -n orcherSecurity Note: In production, use a secrets manager (AWS Secrets Manager, HashiCorp Vault, Kubernetes External Secrets) instead of kubectl secrets.
Step 2: Deploy PostgreSQL
For production, use a managed service (RDS, Cloud SQL). For self-hosting, use this StatefulSet:
1# postgres.yaml2apiVersion: v13kind: Service4metadata:5 name: postgres6 namespace: orcher7spec:8 ports:9 - port: 543210 targetPort: 543211 selector:12 app: postgres13 clusterIP: None # Headless service for StatefulSet14---15apiVersion: apps/v116kind: StatefulSet17metadata:18 name: postgres19 namespace: orcher20spec:21 serviceName: postgres22 replicas: 123 selector:24 matchLabels:25 app: postgres26 template:27 metadata:28 labels:29 app: postgres30 spec:31 containers:32 - name: postgres33 image: postgres:1534 ports:35 - containerPort: 543236 name: postgres37 env:38 - name: POSTGRES_DB39 valueFrom:40 secretKeyRef:41 name: postgres-credentials42 key: database43 - name: POSTGRES_USER44 valueFrom:45 secretKeyRef:46 name: postgres-credentials47 key: username48 - name: POSTGRES_PASSWORD49 valueFrom:50 secretKeyRef:51 name: postgres-credentials52 key: password53 - name: PGDATA54 value: /var/lib/postgresql/data/pgdata55 volumeMounts:56 - name: postgres-storage57 mountPath: /var/lib/postgresql/data58 resources:59 requests:60 memory: "2Gi"61 cpu: "1"62 limits:63 memory: "4Gi"64 cpu: "2"65 livenessProbe:66 exec:67 command:68 - pg_isready69 - -U70 - orcher71 initialDelaySeconds: 3072 periodSeconds: 1073 readinessProbe:74 exec:75 command:76 - pg_isready77 - -U78 - orcher79 initialDelaySeconds: 580 periodSeconds: 581 volumeClaimTemplates:82 - metadata:83 name: postgres-storage84 spec:85 accessModes: [ "ReadWriteOnce" ]86 storageClassName: "fast-ssd" # Use your cluster's fast storage class87 resources:88 requests:89 storage: 100GiApply it:
1kubectl apply -f postgres.yaml23# Wait for it to be ready4kubectl wait --for=condition=ready pod -l app=postgres -n orcher --timeout=300s56# Verify7kubectl get pods -n orcher8# Should show postgres-0 RunningInitialize the database schema:
1# Connect to the database2kubectl exec -it postgres-0 -n orcher -- psql -U orcher34# Run initialization SQL5CREATE TABLE IF NOT EXISTS workflow_queue (6 id UUID PRIMARY KEY DEFAULT gen_random_uuid(),7 workflow_name TEXT NOT NULL,8 input JSONB,9 status TEXT NOT NULL DEFAULT 'pending',10 retry_count INT DEFAULT 0,11 next_retry_at TIMESTAMP,12 created_at TIMESTAMP DEFAULT NOW(),13 updated_at TIMESTAMP DEFAULT NOW(),14 completed_at TIMESTAMP15);1617CREATE INDEX idx_workflow_queue_status ON workflow_queue(status, created_at);18CREATE INDEX idx_workflow_queue_retry ON workflow_queue(next_retry_at) WHERE status = 'pending';1920CREATE TABLE IF NOT EXISTS workflow_execution_journal (21 workflow_id UUID NOT NULL,22 step_id INT NOT NULL,23 task_name TEXT NOT NULL,24 input JSONB,25 output JSONB,26 status TEXT NOT NULL,27 error TEXT,28 created_at TIMESTAMP DEFAULT NOW(),29 PRIMARY KEY (workflow_id, step_id)30);3132CREATE INDEX idx_workflow_execution_journal_lookup ON workflow_execution_journal(workflow_id, step_id);33CREATE INDEX idx_workflow_execution_journal_workflow ON workflow_execution_journal(workflow_id);3435-- Exit36\qStep 3: Build and Push Your Application Image
1# Dockerfile2# Multi-stage build for small final image3FROM rust:1.75 as builder45WORKDIR /app67# Copy dependency manifests8COPY Cargo.toml Cargo.lock ./910# Create dummy main to cache dependencies11RUN mkdir src && \12 echo "fn main() {}" > src/main.rs && \13 cargo build --release && \14 rm -rf src1516# Copy real source code17COPY src ./src1819# Build real application20RUN touch src/main.rs && cargo build --release2122# Final stage - minimal runtime image23FROM debian:bookworm-slim2425# Install runtime dependencies26RUN apt-get update && apt-get install -y \27 ca-certificates \28 libssl3 \29 libpq5 \30 && rm -rf /var/lib/apt/lists/*3132# Copy binary from builder33COPY --from=builder /app/target/release/orcher-app /usr/local/bin/orcher-app3435# Create non-root user36RUN useradd -m -u 1000 orcher3738USER orcher3940EXPOSE 30004142# Health check endpoint43HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \44 CMD curl -f http://localhost:3000/health || exit 14546CMD ["orcher-app"]Build and push:
1# Build2docker build -t your-registry.com/orcher-app:v1.0 .34# Push5docker push your-registry.com/orcher-app:v1.067# Tag as latest8docker tag your-registry.com/orcher-app:v1.0 your-registry.com/orcher-app:latest9docker push your-registry.com/orcher-app:latestStep 4: Deploy Application (API + Workers)
1# orcher-app.yaml2---3# ConfigMap for shared configuration4apiVersion: v15kind: ConfigMap6metadata:7 name: orcher-config8 namespace: orcher9data:10 LOG_LEVEL: "info"11 WORKER_CONCURRENCY: "10"12 POLL_INTERVAL_MS: "100"13 MAX_WORKFLOW_DURATION_SECS: "3600"14---15# API Deployment16apiVersion: apps/v117kind: Deployment18metadata:19 name: orcher-api20 namespace: orcher21 labels:22 app: orcher-api23 component: api24spec:25 replicas: 326 strategy:27 type: RollingUpdate28 rollingUpdate:29 maxSurge: 130 maxUnavailable: 0 # Zero downtime31 selector:32 matchLabels:33 app: orcher-api34 template:35 metadata:36 labels:37 app: orcher-api38 component: api39 spec:40 # Spread across availability zones41 affinity:42 podAntiAffinity:43 preferredDuringSchedulingIgnoredDuringExecution:44 - weight: 10045 podAffinityTerm:46 labelSelector:47 matchExpressions:48 - key: app49 operator: In50 values:51 - orcher-api52 topologyKey: topology.kubernetes.io/zone53 containers:54 - name: api55 image: your-registry.com/orcher-app:latest56 ports:57 - containerPort: 300058 name: http59 protocol: TCP60 env:61 - name: MODE62 value: "api"63 - name: DATABASE_URL64 valueFrom:65 secretKeyRef:66 name: orcher-config67 key: database-url68 envFrom:69 - configMapRef:70 name: orcher-config71 resources:72 requests:73 memory: "512Mi"74 cpu: "500m"75 limits:76 memory: "1Gi"77 cpu: "1000m"78 livenessProbe:79 httpGet:80 path: /health81 port: 300082 initialDelaySeconds: 1083 periodSeconds: 1084 timeoutSeconds: 585 failureThreshold: 386 readinessProbe:87 httpGet:88 path: /ready89 port: 300090 initialDelaySeconds: 591 periodSeconds: 592 timeoutSeconds: 393 failureThreshold: 294---95# Worker Deployment96apiVersion: apps/v197kind: Deployment98metadata:99 name: orcher-worker100 namespace: orcher101 labels:102 app: orcher-worker103 component: worker104spec:105 replicas: 5106 strategy:107 type: RollingUpdate108 rollingUpdate:109 maxSurge: 2110 maxUnavailable: 1111 selector:112 matchLabels:113 app: orcher-worker114 template:115 metadata:116 labels:117 app: orcher-worker118 component: worker119 spec:120 affinity:121 podAntiAffinity:122 preferredDuringSchedulingIgnoredDuringExecution:123 - weight: 100124 podAffinityTerm:125 labelSelector:126 matchExpressions:127 - key: app128 operator: In129 values:130 - orcher-worker131 topologyKey: topology.kubernetes.io/zone132 containers:133 - name: worker134 image: your-registry.com/orcher-app:latest135 env:136 - name: MODE137 value: "worker"138 - name: DATABASE_URL139 valueFrom:140 secretKeyRef:141 name: orcher-config142 key: database-url143 envFrom:144 - configMapRef:145 name: orcher-config146 resources:147 requests:148 memory: "1Gi"149 cpu: "1000m"150 limits:151 memory: "2Gi"152 cpu: "2000m"153 livenessProbe:154 exec:155 command:156 - /bin/sh157 - -c158 - "pgrep -f worker || exit 1"159 initialDelaySeconds: 30160 periodSeconds: 30161 timeoutSeconds: 5162 failureThreshold: 3163---164# Service for API165apiVersion: v1166kind: Service167metadata:168 name: orcher-api169 namespace: orcher170spec:171 type: LoadBalancer172 ports:173 - port: 80174 targetPort: 3000175 protocol: TCP176 name: http177 selector:178 app: orcher-api179---180# Horizontal Pod Autoscaler for Workers181apiVersion: autoscaling/v2182kind: HorizontalPodAutoscaler183metadata:184 name: orcher-worker-hpa185 namespace: orcher186spec:187 scaleTargetRef:188 apiVersion: apps/v1189 kind: Deployment190 name: orcher-worker191 minReplicas: 3192 maxReplicas: 20193 metrics:194 - type: Resource195 resource:196 name: cpu197 target:198 type: Utilization199 averageUtilization: 70200 - type: Resource201 resource:202 name: memory203 target:204 type: Utilization205 averageUtilization: 80206 behavior:207 scaleDown:208 stabilizationWindowSeconds: 300 # Wait 5 min before scaling down209 policies:210 - type: Percent211 value: 50212 periodSeconds: 60213 scaleUp:214 stabilizationWindowSeconds: 0 # Scale up immediately215 policies:216 - type: Percent217 value: 100218 periodSeconds: 30Deploy everything:
1# Apply the manifests2kubectl apply -f orcher-app.yaml34# Watch the rollout5kubectl rollout status deployment/orcher-api -n orcher6kubectl rollout status deployment/orcher-worker -n orcher78# Get the LoadBalancer IP9kubectl get svc orcher-api -n orcher1011# Test the API12export API_URL=$(kubectl get svc orcher-api -n orcher -o jsonpath='{.status.loadBalancer.ingress[0].ip}')13curl http://$API_URL/health14# Should return: {"status": "healthy"}Verification checklist:
1# Check all pods are running2kubectl get pods -n orcher3# Should show:4# - postgres-0 (Running)5# - orcher-api-xxxxx (3 replicas, all Running)6# - orcher-worker-xxxxx (5 replicas, all Running)78# Check logs for errors9kubectl logs -l app=orcher-api -n orcher --tail=10010kubectl logs -l app=orcher-worker -n orcher --tail=1001112# Check database connectivity from app13kubectl exec -it deployment/orcher-api -n orcher -- /bin/sh14# Inside pod:15# Test database connection (if you have psql)Congratulations! Orcher is now running in production.
Next Steps
- Scaling and Performance -- Worker pools, database tuning
- Back to Overview