Overview
Apache Flink is the leading open-source stream processing framework. Unlike Spark Streaming's micro-batch approach, Flink processes events one at a time with exactly-once semantics. It's the go-to choice for stateful stream processing at scale.
Key Features
- ✓ True Streaming: Event-at-a-time processing
- ✓ Stateful Processing: Managed state with checkpoints
- ✓ Flink SQL: Stream processing with SQL
- ✓ Exactly-once: Guaranteed processing semantics
- ✓ Event Time: Proper handling of out-of-order events
- ✓ Batch & Stream: Unified API for both
Pros
- 👍 Best-in-class stream processing
- 👍 Handles complex event processing
- 👍 Strong exactly-once guarantees
- 👍 Mature and battle-tested
- 👍 Good Kafka integration
Cons
- 👎 Complex to operate
- 👎 Steep learning curve
- 👎 Resource intensive
- 👎 Smaller ecosystem than Spark
- 👎 Debugging can be challenging
Best For
Teams with complex real-time processing needs—fraud detection, real-time analytics, complex event processing. When micro-batching isn't enough.
Founded: 2014 HQ: Apache Software Foundation