Email Campaigns with Batching & Rate Limiting
Bulk email sends need to respect provider rate limits while tracking progress. With Orcher, you process recipients in batches, sleep between them using durable timers, and checkpoint progress automatically. If the worker restarts at batch 47 of 100, it skips the first 46 batches (already journaled) and continues from batch 47.
How it works
The loop alternates a send with a one-second durable sleep — and both are journaled. On replay, completed batches return their recorded results and completed sleeps finish instantly, so a restart never re-sends an email and never re-waits a second that already passed.
Example: a rate-limited campaign
1import {2 Workflow,3 Tasks,4 Task,5 WorkflowContext,6 createTaskRefs,7} from '@orcher/sdk';89@Tasks()10class EmailTasks {11 @Task()12 async fetchRecipients(segmentId: string): Promise<Recipient[]> {13 return await db.queryRecipientsBySegment(segmentId);14 }1516 @Task()17 async sendEmailBatch(batch: Recipient[]): Promise<BatchResult> {18 let sent = 0;19 for (const recipient of batch) {20 await emailProvider.send(recipient.email, recipient.template);21 sent++;22 }23 return { sent, failed: batch.length - sent };24 }2526 @Task()27 async recordCampaignStats(stats: CampaignStats): Promise<void> {28 await analytics.record(stats);29 }30}3132const tasks = createTaskRefs(EmailTasks);3334@Workflow({ name: 'send_email_campaign', version: '1.0.0' })35class SendEmailCampaign {36 async run(37 ctx: WorkflowContext,38 input: CampaignInput,39 ): Promise<CampaignResult> {40 const recipients = await ctx.executeTask(41 tasks.fetchRecipients, input.segmentId,42 );43 const total = recipients.length;44 let sent = 0;45 let failed = 0;4647 // Process in batches of 100 with rate limiting48 for (let i = 0; i < recipients.length; i += 100) {49 const batch = recipients.slice(i, i + 100);5051 try {52 const result = await ctx.executeTask(tasks.sendEmailBatch, batch);53 sent += result.sent;54 } catch {55 failed += batch.length;56 }5758 // Rate limit: 1 second between batches (durable timer)59 await ctx.sleep(1000);60 }6162 await ctx.executeTask(tasks.recordCampaignStats, {63 campaignId: input.campaignId,64 total,65 sent,66 failed,67 });6869 return { total, sent, failed };70 }71}The ctx.sleep() call is a durable timer — it's journaled, so on replay it completes instantly rather than sleeping again. This means a 100-batch campaign that crashes at batch 50 doesn't re-wait the 49 seconds of sleep that already passed.
Where the journal can't see
The checkpoint granularity is the task boundary — and this example loops over 100 individual sends inside one task. If the task dies at email 51, its retry starts the loop from email 1:
Two defenses, use both: pass a provider idempotency key per recipient (message ID derived from campaign + recipient) so a re-sent email deduplicates at the provider, and keep batches small enough that a worst-case replay is acceptable.
Design notes
- Batch size is also blast radius. It sets checkpoint granularity (as in batch processing) and the maximum duplicate exposure if a mid-batch failure meets a non-idempotent provider. 100 is a reasonable default; size down if your provider has no deduplication.
- The sleep sets a pace, not a precise rate. Actual throughput is
batch_size / (send duration + 1s)— the timer adds spacing but doesn't measure the provider's clock. For strict provider budgets, size batches conservatively rather than tuning the sleep. - Failed batches are counted, not retried — deliberately. The workflow records them in the stats and finishes, giving you a re-run worklist instead of a stuck campaign. If a batch deserves in-place retries, wrap the send in retry with backoff.
- Fetching recipients once pins the audience. The recipient list is journaled with
fetch_recipients, so a resume six hours later emails the original segment — subscribers added mid-run aren't silently included. That's usually what you want; re-fetch per batch only if you explicitly want a live audience.