Getting Started

Learning Path: Data Engineers


Orcher gives data engineers durable execution backed by PostgreSQL — no message broker and no separate scheduler to run. Every step in your pipeline is automatically journaled, so if a server crashes at record 847,392 of a million-row import, execution resumes at 847,393. Completed tasks are replayed from the journal rather than re-executed, so recovery never repeats finished work — make external writes idempotent so a task retried before recording its result stays safe. This guide walks you through the patterns that matter most for data engineering: batch processing, incremental sync, CDC, parallel fan-out, and production error handling.

Prerequisites Basic SQL, any programming language
Time to complete 2–3 hours
You'll build A crash-safe, parallel ETL pipeline

The path

Four parts, each building on the last — from a first checkpointed pipeline to a production deployment. Follow them in order, or jump to the stage that matches what you're building.

Every pattern here has a reference companion in the Data Pipelines & ETL use-case guide — this path is the guided walk-through; that section is the deep dive to return to.