Spark Streaming
Apache Spark's stream processing engine providing unified batch and streaming analytics with exactly-once semantics.
✨ Key Features
- ✓ Same API: DataFrame/Dataset for batch and streaming
- ✓ Code reuse: Write once, run batch or streaming
- ✓ Consistent semantics: Same results either way
- ✓ Easy migration: Convert batch to streaming easily
- ✓ Declarative API: SQL-like operations
- ✓ Event time: Watermarks and late data handling
- ✓ Exactly-once: End-to-end guarantees
- ✓ Continuous processing: Sub-millisecond latency mode
- ✓ Micro-batch: Default, ~100ms latency
- ✓ Continuous: Experimental ~1ms latency
- ✓ Triggers: Once, processing time, available-now
- ✓ Kafka: Native connector
- ✓ Kinesis: AWS streaming
- ✓ Event Hubs: Azure streaming
- ✓ Files: S3, HDFS, cloud storage
