Amazon Data Firehose (formerly Amazon Kinesis Data Firehose)
Kinesis (aka Kinesis Data Streams)
Kinesis Features (Focus, Data Storage, Scaling, Processing, Replay, Use Cases)
Focus: Real-time data ingestion, processing, and analysis.
Data Storage: Stores data for a configurable period (default 24 hours, up to 7 days).
Scaling: Requires manual scaling through shard management.
Processing: Allows for custom processing with sub-second latency.
Replay: Supports replay capability.
Use Cases: Building real-time dashboards, processing clickstream data, and building personalized recommendations.
Kinesis Data Firehose Features (Focus, Data Storage, Scaling, Processing, Replay, Use Cases)
Focus: Delivering streaming data to various destinations (S3, Redshift, Elasticsearch, etc.).
Data Storage: Does not store data; delivers directly to specified destinations.
Scaling: Fully managed, automatic scaling.
Processing: Near real-time processing with configurable buffer sizes and intervals.
Replay: Does not support replay capability.
Use Cases: Loading data into data warehouses, building data lakes, and sending data to analytics platforms.
Lex
Service for building conversational interfaces for applications using voice and text.
Polly
Amazon Polly is a fully-managed service that generates voice on demand, converting any text to an audio stream.
Rekognition
Identifies a wide range of objects, scenes, and activities within images and videos.
Comprehend
NLP service that uses machine learning to extract insights from text
EMR
Elastic Map Reduce
Redshift
OLAP
OLTP
Lake Formation
MSK
Managed Streaming for Apache Kafka
Redshift Streaming Ingestion
Redshift Spectrum
Redshift Spectrum is a Redshift feature that allows you to query data in Amazon S3 without loading them into Redshift tables. Redshift Spectrum is not capable of moving data from S3 to Redshift.
Why isn’t Amazon Data Firehose the lowest latency solution?
It stages the data in S3 first then copies it to the final destination
FSx for Lustre
a fully managed, high-performance file system optimized for compute-intensive workloads like high-performance computing (HPC), machine learning, and video processing
Amazon IQ
a freelancing platform designed to help customers quickly find, engage, and pay AWS Certified third-party experts for on-demand project work
DeepLens
RTSP
Kinesis Video Streams
Lambda Functions
-Serverless and event -driven
- Execute single discrete tasks in response to events such as image upload to S3 or a new entry in a DynamoDB table
- Max excution duration of 15 min. Not suitable for long-running jobs like ETL.
- Effective for simple, short-lived tasks
Step Functions