Operations

Caching & Schema


In-Memory Cache

The orchestrator uses a Moka-based LRU cache (HighPerfCache) to reduce database roundtrips for hot workflow state. This is an in-process cache — no external dependency required.

Cache Tiers

CacheMax EntriesTTLPurpose
Workflow state10,0005 minActive workflow execution data
Journal entries5,0001 minRecent journal entries for replay
Claims50,0002 minWorkflow claim ownership (shorter than claim timeout)

How It Works

The HighPerfCache sits between the dispatch engine and PostgreSQL:

  1. On poll — Check cache for workflow state before querying the database
  2. On update — Invalidate cached entry so the next read fetches fresh data
  3. On eviction — LRU eviction when cache is full; no data loss since PostgreSQL is authoritative

Cache metrics (hits, misses, invalidations) are tracked for monitoring.

Memory Impact

Concurrent WorkflowsApproximate Cache Memory
Up to 100~50 MB
100–1,000~200 MB
1,000–5,000~500 MB
5,000–10,000~1 GB

Memory usage scales with the number of active (not total) workflows.


Database Schema Details

Workflow Claiming

The workflow_executions table uses claim-based polling for safe distributed work assignment:

1
-- Workers claim workflows with SELECT FOR UPDATE SKIP LOCKED
2
-- This avoids contention between multiple orchestrator nodes
3
SELECT * FROM workflow_executions
4
WHERE task_queue = $1
5
AND namespace = $2
6
AND status IN ('pending', 'running')
7
AND claimed_at IS NULL
8
ORDER BY priority DESC, submitted_at ASC
9
FOR UPDATE SKIP LOCKED
10
LIMIT 1;

Key columns: claimed_by, claim_timeout_at, pending_requeue, shard_id, priority.

Journal (Event Sourcing)

Every workflow step is recorded in the journal for replay protection:

1
-- Journal entries for a workflow execution
2
SELECT * FROM workflow_execution_journal
3
WHERE execution_id = $1
4
ORDER BY sequence_number ASC;

Entry types include: WorkflowStarted, StepCompleted, TimerStarted, TimerFired, EventReceived, WorkflowCompleted, WorkflowFailed.

Outbox Pattern

Task dispatch uses the outbox pattern for atomic delivery:

1
-- State change + outbox insert in one transaction
2
BEGIN;
3
UPDATE workflow_executions SET status = 'running' WHERE id = $1;
4
INSERT INTO outbox_queue (execution_id, task_queue, namespace, payload)
5
VALUES ($1, $2, $3, $4);
6
COMMIT;
7
8
-- OutboxProcessor claims entries with lease-based locking
9
UPDATE outbox_queue
10
SET status = 'processing', lease_expires_at = NOW() + INTERVAL '30 seconds'
11
WHERE status = 'pending'
12
LIMIT 10
13
RETURNING *;

Next Steps