Amazon Kinesis Data Streams

Managed service for ingesting real-time streaming data and keeping it available for multiple consumers to read and re-read.

Verified against AWS documentation · 2026-09-09

Key facts

01Reading doesn't delete — data stays for the full retention window, so many consumers can read the same stream independently
02Shard = the throughput unit: 1 MB/s or 1,000 records/s in, 2 MB/s out; records land in order within a shard
03Provisioned mode = you choose the shard count; On-demand mode = AWS scales shards for you
04Limits: 1 MB max record size, 24-hour default retention (extendable to 365 days), partition key decides the shard

Phrases and scenarios that point here.

"Real time", "streaming", "clickstream", "IoT telemetry", "within seconds"
Multiple teams or applications need the same events for different purposes → Data Streams, not SQS
Must be able to replay or reprocess data after a failure → Data Streams retention
Order matters per device/user/session → partition key keeps those records on one shard

Just land the stream in S3, Redshift, or OpenSearch with no code → Amazon Data Firehose
One message, one consumer, then it's gone → SQS
Windowed aggregations or SQL over the stream → Amazon Managed Service for Apache Flink

01Clickstream on a retail site where one consumer drives a live traffic dashboard and another archives to S3 — same stream, two readers, neither blocking the other
02Fleet telemetry from delivery bikes, using bike ID as the partition key so each bike's GPS pings stay in order even as the fleet grows
03Real-time fraud scoring on payment events, where a scoring consumer reacts in seconds and a nightly batch job re-reads the same records for reporting
04Log ingestion for an app too chatty for a single processor — fan out to a Lambda consumer for alerting and a Firehose consumer for the data lake
05Recovering from a bad consumer deploy: because records are retained, you rewind and reprocess the last few hours instead of losing them

01Provisioned mode: hourly rate per shard, plus PUT payload units metered in 25 KB chunks — a 1 KB record still burns a whole unit, so batching small records cuts cost sharply
02On-demand mode: hourly rate per stream, plus GB written and GB read — no shard math, higher unit price, better for spiky or unknown traffic
03Shards bill around the clock whether or not data flows — an over-provisioned idle stream is the classic Kinesis bill surprise
04The first 24 hours of retention is included; extended retention (to 7 days) and long-term retention (to 365 days) are separate line items
05Enhanced fan-out, which gives each consumer its own 2 MB/s per shard, bills per consumer-shard hour plus GB retrieved — it's an opt-in add-on, not the default