Getting Started

Email Campaigns with Batching & Rate Limiting


Bulk email sends need to respect provider rate limits while tracking progress. With Orcher, you process recipients in batches, sleep between them using durable timers, and checkpoint progress automatically. If the worker restarts at batch 47 of 100, it skips the first 46 batches (already journaled) and continues from batch 47.

How it works

The loop alternates a send with a one-second durable sleep — and both are journaled. On replay, completed batches return their recorded results and completed sleeps finish instantly, so a restart never re-sends an email and never re-waits a second that already passed.

Resume Without Re-Sending or Re-Waiting
✓
send_email_batch(046)100 sent — journaled
⏱
sleep(1s)Completed — journaled
✓
send_email_batch(047)100 sent — journaled
✗
worker crashesMid-campaign, 53 batches to go
↻
replay47 batches + 47 sleeps replay instantly — nothing re-sent, nothing re-waited
✓
send_email_batch(048)The campaign continues on cadence
No subscriber hears from you twice; the provider's rate limit is never burst on resume.

Example: a rate-limited campaign

1
import {
2
Workflow,
3
Tasks,
4
Task,
5
WorkflowContext,
6
createTaskRefs,
7
} from '@orcher/sdk';
8
9
@Tasks()
10
class EmailTasks {
11
@Task()
12
async fetchRecipients(segmentId: string): Promise<Recipient[]> {
13
return await db.queryRecipientsBySegment(segmentId);
14
}
15
16
@Task()
17
async sendEmailBatch(batch: Recipient[]): Promise<BatchResult> {
18
let sent = 0;
19
for (const recipient of batch) {
20
await emailProvider.send(recipient.email, recipient.template);
21
sent++;
22
}
23
return { sent, failed: batch.length - sent };
24
}
25
26
@Task()
27
async recordCampaignStats(stats: CampaignStats): Promise<void> {
28
await analytics.record(stats);
29
}
30
}
31
32
const tasks = createTaskRefs(EmailTasks);
33
34
@Workflow({ name: 'send_email_campaign', version: '1.0.0' })
35
class SendEmailCampaign {
36
async run(
37
ctx: WorkflowContext,
38
input: CampaignInput,
39
): Promise<CampaignResult> {
40
const recipients = await ctx.executeTask(
41
tasks.fetchRecipients, input.segmentId,
42
);
43
const total = recipients.length;
44
let sent = 0;
45
let failed = 0;
46
47
// Process in batches of 100 with rate limiting
48
for (let i = 0; i < recipients.length; i += 100) {
49
const batch = recipients.slice(i, i + 100);
50
51
try {
52
const result = await ctx.executeTask(tasks.sendEmailBatch, batch);
53
sent += result.sent;
54
} catch {
55
failed += batch.length;
56
}
57
58
// Rate limit: 1 second between batches (durable timer)
59
await ctx.sleep(1000);
60
}
61
62
await ctx.executeTask(tasks.recordCampaignStats, {
63
campaignId: input.campaignId,
64
total,
65
sent,
66
failed,
67
});
68
69
return { total, sent, failed };
70
}
71
}

The ctx.sleep() call is a durable timer — it's journaled, so on replay it completes instantly rather than sleeping again. This means a 100-batch campaign that crashes at batch 50 doesn't re-wait the 49 seconds of sleep that already passed.

Where the journal can't see

The checkpoint granularity is the task boundary — and this example loops over 100 individual sends inside one task. If the task dies at email 51, its retry starts the loop from email 1:

Inside a Task, You're on Your Own
1
send_email_batch(012)Task starts — 100 recipients
✓
emails 1–50 sentProvider accepted them — no journal entries
✗
send #51 failsTask errors — nothing was checkpointed
↻
task retriesThe loop starts from email 1
✗
emails 1–50 sent againDuplicates — unless sends carry idempotency keys
The journal checkpoints between tasks, never inside them — design the inside accordingly.

Two defenses, use both: pass a provider idempotency key per recipient (message ID derived from campaign + recipient) so a re-sent email deduplicates at the provider, and keep batches small enough that a worst-case replay is acceptable.

Design notes

  • Batch size is also blast radius. It sets checkpoint granularity (as in batch processing) and the maximum duplicate exposure if a mid-batch failure meets a non-idempotent provider. 100 is a reasonable default; size down if your provider has no deduplication.
  • The sleep sets a pace, not a precise rate. Actual throughput is batch_size / (send duration + 1s) — the timer adds spacing but doesn't measure the provider's clock. For strict provider budgets, size batches conservatively rather than tuning the sleep.
  • Failed batches are counted, not retried — deliberately. The workflow records them in the stats and finishes, giving you a re-run worklist instead of a stuck campaign. If a batch deserves in-place retries, wrap the send in retry with backoff.
  • Fetching recipients once pins the audience. The recipient list is journaled with fetch_recipients, so a resume six hours later emails the original segment — subscribers added mid-run aren't silently included. That's usually what you want; re-fetch per batch only if you explicitly want a live audience.

Next Steps