Mage
Modern open-source data pipeline tool combining notebooks, orchestration, and observability for building and managing data workflows.
✨ Key Features
- ✓ Interactive blocks: Notebook-style code cells
- ✓ Instant feedback: Run and see results immediately
- ✓ Version control: Git-friendly file structure
- ✓ Testing: Built-in unit and integration tests
- ✓ DAG visualization: Visual dependency graphs
- ✓ Scheduling: Cron and event-based triggers
- ✓ Backfills: Historical data processing
- ✓ Streaming: Real-time pipeline support
- ✓ Sources: Databases, APIs, cloud storage
- ✓ Transformations: SQL, Python, R blocks
- ✓ Destinations: Warehouses, lakes, APIs
- ✓ dbt integration: Native dbt model support
- ✓ Monitoring: Pipeline health dashboards
- ✓ Alerting: Failure notifications
- ✓ Logging: Structured pipeline logs
- ✓ Lineage: Data dependency tracking
📁 More Orchestration Tools
Apache Airflow
The industry-standard workflow orchestration platform
Argo Workflows
Dagster
Cloud-native orchestration platform for data pipelines
dbt Cloud