DynamoDB Streams: Change Data Capture Explained

DynamoDB Streams is a change data capture (CDC) feature: every insert, update, and delete on a table is recorded as a time-ordered log you can consume. It’s the foundation for replication, real-time aggregation, search indexing, and event-driven architectures built on DynamoDB.

This guide explains stream view types, how ordering and shards work, the Lambda trigger model, and the patterns Streams unlocks.

What a stream record contains

When you enable Streams on a table, each item-level write produces a stream record with an eventName of INSERT, MODIFY, or REMOVE, plus the item’s keys and — depending on the view type — its data before and/or after the change.

{
  "eventName": "MODIFY",
  "dynamodb": {
    "Keys": { "PK": { "S": "ORDER#A1" } },
    "OldImage": { "status": { "S": "PENDING" } },
    "NewImage": { "status": { "S": "SHIPPED" } },
    "SequenceNumber": "0700000000000123456",
    "ApproximateCreationDateTime": 1750000000
  }
}

Stream view types

You choose how much data each record carries when you enable the stream. This is fixed per stream and can’t be changed without disabling and re-enabling.

NEW_AND_OLD_IMAGES is the most useful for general CDC — you can compute exactly what changed. Choose KEYS_ONLY when consumers re-fetch the current item themselves and you want smaller records. For deletes (including TTL expirations), only OLD_IMAGE carries the item’s last state.

Shards and ordering

A stream is divided into shards, which map to the table’s underlying partitions and split/merge as the table scales. The critical guarantee:

• Per–partition key, records are strictly ordered. Every change to a single item arrives in the exact sequence it happened.
• There is no ordering across different partition keys. Two unrelated items’ changes can be processed in any relative order.

This is usually exactly what you want: operations on the same entity are serialized, while independent entities process in parallel. Design consumers so correctness never depends on cross-item global ordering.

Consuming with Lambda triggers

The most common consumer is an event source mapping to AWS Lambda. Lambda polls the stream, batches records per shard, and invokes your function:

def handler(event, context):
    for record in event["Records"]:
        if record["eventName"] == "MODIFY":
            new = record["dynamodb"]["NewImage"]
            old = record["dynamodb"]["OldImage"]
            # react to the change

Key behaviors of the Lambda integration:

• Ordering within a shard is preserved; Lambda processes one batch per shard at a time (subject to your parallelization factor).
• At-least-once delivery. A record can be delivered more than once, so make handlers idempotent — key your side effects on SequenceNumber or an item attribute.
• Failure blocks the shard. By default, a failing batch is retried and holds up that shard until it succeeds or the records expire. Use BisectBatchOnFunctionError, MaximumRetryAttempts, and an on-failure destination (SQS/SNS DLQ) so a poison record doesn’t stall the partition.

Patterns Streams unlock

Replication and fan-out

Project changes into a second table, an OpenSearch index, or a downstream service. Materialized views — a read-optimized copy shaped for a different access pattern — are a staple of this.

Aggregation and counters

Maintain running totals (orders per customer, sum of line items) by reacting to INSERT/MODIFY/REMOVE. This keeps expensive aggregates out of the hot write path. For atomic counter updates inside a single change, transactions may be a better fit; Streams suit cross-item or eventually-consistent rollups.

TTL-driven cleanup and archival

TTL deletions flow through the stream as REMOVE records tagged with userIdentity.principalId = "dynamodb.amazonaws.com". A consumer can archive the expiring item to S3 or cascade-delete children — see TTL for the full pattern.

Audit logs

Every MODIFY with NEW_AND_OLD_IMAGES is a ready-made before/after audit entry. Append them to an immutable store.

Retention and the 24-hour limit

Stream records live for 24 hours, then disappear. This makes Streams a processing pipeline, not a durable event store. If a consumer falls behind by more than a day, it permanently loses those records.

Operationally:

• Monitor IteratorAge (how far behind the consumer is). A climbing iterator age means you’re at risk of data loss.
• Alarm on it and on consumer errors. A dead event source mapping silently stops processing — exactly the kind of failure that surfaces only when downstream data is already stale.
• If you need long retention or multiple independent consumers, use the Kinesis Data Streams option for DynamoDB instead of (or alongside) native Streams.

Costs

You’re billed per stream read request unit (one unit returns up to 4 KB from a shard). Writing to the stream is free; you pay to consume. Lambda invocations and any downstream services (S3, OpenSearch) are billed separately. For most workloads Streams cost is small relative to the table itself, but high-throughput tables with chatty consumers should keep an eye on read units.

A note on operating Streams

The hardest part of running Streams in production isn’t the code — it’s observing the pipeline: which shard is stalled, what the offending record looks like, and whether the consumer is keeping up. Being able to open the source table and inspect the exact item that produced a problematic stream record shortens that loop considerably. A native GUI like Tablyne is handy here for reading the live item alongside the stream record your handler choked on.

Streams turn DynamoDB into an event source you can build reactive systems on. Pick the right view type up front, make consumers idempotent, and treat the 24-hour retention window as a hard operational constraint — see best practices for more on keeping consumers healthy.