Apache Hudi
Open-source data lake framework providing streaming data lake capabilities with ACID transactions, upserts, and incremental processing.
✨ Key Features
- ✓ Upserts: Insert or update based on key
- ✓ Deletes: Soft and hard deletes
- ✓ Merge on read: Lazy reconciliation
- ✓ Copy on write: Immediate consistency
- ✓ Near real-time: Minutes latency
- ✓ Kafka integration: Direct streaming ingest
- ✓ DeltaStreamer: Built-in ingestion utility
- ✓ Flink support: Stream processing integration
- ✓ Change streams: Track data changes
- ✓ Incremental queries: Process only deltas
- ✓ Time travel: Query historical versions
- ✓ Rollback: Revert to previous state
- ✓ Atomicity: All-or-nothing commits
- ✓ Consistency: Schema enforcement
- ✓ Isolation: Concurrent reader/writer
- ✓ Durability: Committed data persists
📁 More Data Lakes Tools
Delta Lake
Open-source storage layer bringing reliability to data lakes
Dremio
Lakehouse platform with SQL query engine
Apache Iceberg
Open table format for huge analytic datasets
Onehouse