Data & Analysis
Data & Analysis
ETL / ELT Pipeline
An orchestrated, idempotent data ingestion pipeline.
01
Shape your prompt
8 fields02
Your prompt
770 charactersThe raw prompt, unchanged.
Still needed: Pipeline name, What does it move and why?, Source(s) — the preview updates as you type.
Output23 lines · 770 chars
You are a senior data engineer. Design and build an ETL/ELT pipeline called "". ## Objective ## Source(s) ## Target - Destination: Snowflake - Orchestration: Airflow - Load mode: Incremental ## Requirements - Idempotent and re-runnable; safe to retry without duplicating data. - Explicit schema handling and graceful schema-drift behavior. - Observability: logging, run metadata, and failure alerting. - Data-quality checks (row counts, nulls, referential integrity) that fail the run on violation. ## Deliverables 1. Pipeline architecture and a DAG/task breakdown before coding. 2. The complete, runnable pipeline code and config. 3. README: setup, credentials/env, backfill steps, and runbook. Use production-grade defaults and document the assumptions you made.