Change Data Capture with Debezium and Kafka

Change Data Capture (CDC) streams row-level database changes into downstream systems by reading the database’s transaction log. Unlike batch ETL, CDC provides low-latency, ordered event streams without changing application code. Debezium is an open-source platform that implements CDC as a set of Kafka Connect source connectors. Running on Apache Kafka, it produces durable, replayable change streams for event-driven architectures.

This is a production operational reference for platform engineers and SREs running Debezium on Kafka Connect: connector configuration, snapshot strategies, schema evolution, topic routing, monitoring, rebalancing, troubleshooting, performance tuning, and security. Property and metric names are aligned with Debezium 2.x/3.x; where behavior changed across versions, that is called out explicitly.

How Debezium Captures Database Changes

Debezium reads each source database’s transaction log. The mechanism varies by database:

• MySQL: Reads the binary log (binlog) as a replication client.
• PostgreSQL: Uses logical decoding via a replication slot and an output plugin (pgoutput is the standard built-in plugin).
• MongoDB: Uses change streams (the legacy oplog-tailing implementation was removed in Debezium 2.x).
• SQL Server: Reads from SQL Server’s built-in CDC change tables.

In each case, Debezium translates log entries into structured change events and writes them to Kafka topics.

The Change Event Envelope

Every Debezium change event is a Kafka record with a key (typically the source row’s primary key) and a value carrying the change payload. The value follows a standard envelope:

{
  "schema": { "...": "..." },
  "payload": {
    "before": { "id": 1004, "name": "Anne", "email": "anne@example.com" },
    "after": { "id": 1004, "name": "Anne Marie", "email": "anne@example.com" },
    "source": {
      "version": "2.5.0.Final",
      "connector": "postgresql",
      "name": "dbserver1",
      "ts_ms": 1699200000000,
      "snapshot": "false",
      "db": "inventory",
      "schema": "public",
      "table": "customers",
      "txId": 582,
      "lsn": 24023128
    },
    "op": "u",
    "ts_ms": 1699200000500
  }
}

The op field is the operation: c (create), u (update), d (delete), r (read, emitted for snapshot rows), or t (truncate). The source block carries database-specific metadata including the transaction ID and log position (lsn for PostgreSQL, binlog file/position for MySQL). Consumers use this envelope to reconstruct state, detect deletions, and implement event-sourcing patterns.

Two envelope details matter operationally. First, before is fully populated only when the source provides the full old row — PostgreSQL requires REPLICA IDENTITY FULL on the table, otherwise before holds only the primary-key columns. Second, on a delete the after field is null and before carries the last-known row; a downstream that keys only on after will silently drop deletes.

When integrated with a schema registry, the envelope schema is registered as a versioned artifact. Schema Evolution and Compatibility in Confluent Schema Registry governs how subsequent schema changes are handled.

Connector Configuration and Deployment

A Debezium connector is deployed by submitting JSON to the Kafka Connect REST API. Here is a production-oriented PostgreSQL connector configuration:

{
  "name": "inventory-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "pg-primary.internal",
    "database.port": "5432",
    "database.user": "debezium",
    "database.password": "${file:/etc/connector-secrets/pg.properties:password}",
    "database.dbname": "inventory",
    "topic.prefix": "dbserver1",
    "table.include.list": "public.customers,public.orders",
    "plugin.name": "pgoutput",
    "publication.autocreate.mode": "filtered",
    "slot.name": "debezium_inventory",
    "snapshot.mode": "initial",
    "heartbeat.interval.ms": "5000",
    "heartbeat.action.query": "INSERT INTO public.debezium_heartbeat (id, ts) VALUES (1, NOW()) ON CONFLICT (id) DO UPDATE SET ts=EXCLUDED.ts",
    "max.batch.size": "2048",
    "max.queue.size": "8192",
    "poll.interval.ms": "500",
    "tombstones.on.delete": "true",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter.schema.registry.url": "http://schema-registry:8081",
    "value.converter.schema.registry.url": "http://schema-registry:8081"
  }
}

Note that topic.prefix replaces the older database.server.name, which was deprecated in Debezium 1.9 and removed in 2.x. slot.name pins the replication slot name so a restarted connector reattaches to the same slot rather than orphaning the old one.

There are two REST endpoints for creating or updating a connector:

• POST /connectors with the full {"name": ..., "config": {...}} envelope above (creates only; fails if the connector exists).
• PUT /connectors/{name}/config with just the inner config map (the value of config, not the envelope). This is idempotent: it creates the connector if absent and updates it otherwise, which makes it the preferred call for GitOps-style management.

curl -s -X PUT -H "Content-Type: application/json" \
  http://connect-cluster:8083/connectors/inventory-connector/config \
  --data-binary @inventory-connector.config.json

where inventory-connector.config.json contains only the inner config object. The Connect framework distributes tasks across workers. Distributed mode is the only viable option for production; standalone mode is for development and testing. See Kafka Connect Architecture and Deployment Modes for details.

Snapshot Modes

The snapshot.mode property controls how Debezium initializes its view of the source database. The values below are the current Debezium 2.x/3.x set:

There is no exported mode in current Debezium. (An exported PostgreSQL snapshot mode existed in early releases and was removed; it was never a MySQL-specific FLUSH TABLES ... FOR EXPORT mode.)

Choosing a mode comes down to one question — do you need the existing rows, and can you afford to copy them now?

• You need a full backfill of existing data: initial. The default, and correct for nearly all first deployments.
• You only want changes from now on (the historical state already lives elsewhere, or the table is treated as an append-only event source): never. Be deliberate — there is no going back without a re-snapshot, and downstream state will be incomplete for any row not touched after cutover.
• You restored a database or rebuilt a connector and only the schema history is stale, not the data: recovery (schema-history connectors only). It rebuilds DDL state without re-copying rows.
• You are migrating connector config but the offset is intact: no_data re-establishes schema without re-emitting READ events, useful when adding a table you will backfill separately.
• Avoid always in production unless the source is small and you genuinely want a clean re-copy on every restart — a routine pod restart will otherwise trigger a full table scan and a flood of r events.

For large PostgreSQL databases, the initial snapshot opens a REPEATABLE READ transaction and reads all matching rows. This can be long-running but does not block writes. For MySQL, the snapshot’s locking behavior is controlled separately by snapshot.locking.mode (minimal, minimal_percona, extended, or none); minimal holds the global read lock only long enough to read the binlog position and schema, then releases it for the row copy. For very large tables, prefer incremental snapshots (see Performance Tuning) to avoid long snapshot transactions entirely.

Schema Management and Evolution

Debezium generates a schema for every captured table and registers it in the schema registry. With the default TopicNameStrategy, the subjects are <topicPrefix>.<schemaName>.<tableName>-value and <topicPrefix>.<schemaName>.<tableName>-key — for example, dbserver1.public.customers-value.

When a DDL change occurs on the source table (adding a column, altering a type, dropping a column), Debezium emits records carrying a new schema version. Whether the registry accepts it depends on the configured compatibility mode. The Schema Registry & Connect Ecosystem provides the infrastructure, and Schema Evolution and Compatibility in Confluent Schema Registry defines the rules.

Schema History (MySQL, MariaDB, SQL Server)

Connectors that read a database-native DDL stream (MySQL, MariaDB, SQL Server) persist a schema history so they can interpret older log entries after a restart. Configure it via:

schema.history.internal.kafka.topic=schemahistory.inventory
schema.history.internal.kafka.bootstrap.servers=kafka-broker-1:9092,kafka-broker-2:9092
schema.history.internal.kafka.recovery.poll.interval.ms=5000

The schema-history topic must be a single-partition, infinite-retention (compaction off) topic that no other connector shares. PostgreSQL and MongoDB do not use these properties: PostgreSQL receives relation metadata inline via logical-decoding relation messages, so there is no separate history topic.

Handling Breaking Schema Changes

A column drop yields a schema that is backward-incompatible by default. If the registry enforces BACKWARD compatibility, registration of the new schema fails and the converter raises a registration error that fails the task. When a breaking change is unavoidable, options include:

1. Temporarily relax the registry compatibility mode to NONE, register the new schema, then restore the original mode.
2. Plan changes so they are compatible under the active mode — e.g., adding nullable/defaulted columns is backward-compatible. Note that FULL_TRANSITIVE still rejects a column drop.
3. Route the affected table to a new topic with a Single Message Transform (SMT) and migrate downstream consumers.

Topic Routing and Single Message Transforms

By default, Debezium writes each captured table to a topic named <topicPrefix>.<schemaName>.<tableName>. SMTs modify records inside the Connect pipeline before they reach Kafka.

The RegexRouter SMT renames topics:

"transforms": "route",
"transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.route.regex": "dbserver1\\.public\\.(.*)",
"transforms.route.replacement": "cdc.inventory.$1"

This routes all public tables to topics prefixed with cdc.inventory.. To chain SMTs — for example, drop delete events and then route:

"transforms": "filterDeletes,route",
"transforms.filterDeletes.type": "io.debezium.transforms.Filter",
"transforms.filterDeletes.language": "jsr223.groovy",
"transforms.filterDeletes.condition": "value.op != 'd'",
"transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.route.regex": "dbserver1\\.public\\.(.*)",
"transforms.route.replacement": "cdc.inventory.$1"

The Debezium Filter SMT keeps records whose condition evaluates true (here, every non-delete). It needs a JSR-223 script engine on the classpath — Debezium ships none, so add e.g. the Groovy engine (groovy, groovy-jsr223) yourself. SMTs run synchronously on the Connect task thread; heavy scripting or large payloads degrade throughput. For substantial transformation logic, use Kafka Streams or ksqlDB downstream.

Monitoring and Health Checks

Kafka Connect exposes JMX metrics for connector and task status, source-record rates, and offset-commit latency under the kafka.connect domain. Debezium adds its own metrics under the debezium.<plugin> domain (e.g. debezium.postgres, debezium.mysql) for snapshot progress, streaming lag, and slot/log state.

Critical Metrics

Debezium MBeans follow the format debezium.<plugin>:type=connector-metrics,context=<context>,server=<topic.prefix>, where <context> is snapshot or streaming (the MySQL streaming context is historically named binlog).

For MySQL, replace context=streaming with context=binlog on older connector versions. MilliSecondsBehindSource is the lag attribute (there is no SecondsBehindMaster attribute on Debezium MBeans). PostgreSQL replication-slot lag is best monitored from the database itself (see Troubleshooting).

A subtle trap: MilliSecondsBehindSource is computed from the difference between the event’s source timestamp and the connector’s wall clock, so it is meaningful only while events are flowing. On an idle source it stays flat at its last value rather than rising — which is why you also watch MilliSecondsSinceLastEvent and the database-side slot lag. The two together distinguish “connector is behind” from “source is quiet.”

A Prometheus JMX Exporter rule set for these beans:

rules:
  - pattern: "debezium.([^<]+)<type=connector-metrics, context=([^,]+), server=([^>]+)><>([^:]+):"
    name: "debezium_$2_$4"
    labels:
      plugin: "$1"
      context: "$2"
      server: "$3"
  - pattern: "kafka.connect<type=connector-task-metrics, connector=([^,]+), task=([^>]+)><>([^:]+):"
    name: "kafka_connect_task_$3"
    labels:
      connector: "$1"
      task: "$2"

(Adjust spacing to match the exact ObjectName rendering your JVM produces; the exporter matches the flattened domain<key=value, ...><>Attribute form.)

Health Check Endpoints

Query the Connect REST API for connector and task status:

curl -s http://connect-cluster:8083/connectors/inventory-connector/status | jq '.'

A healthy response shows RUNNING for the connector and every task. A Debezium source connector runs a single task, so a FAILED task means the connector is not producing — there is no “reduced parallelism” fallback. Automate restarts with a watchdog, but always investigate the root cause: persistent failures usually mean schema incompatibilities, network issues, purged logs, or database permission problems.

Handling Rebalancing and Task Distribution

In distributed mode, Connect workers use a group-membership protocol to assign connectors and tasks. When a worker joins or leaves, a rebalance occurs. The default incremental cooperative protocol (connect.protocol=sessioned/compatible) only moves the affected tasks rather than stopping the whole group, but a Debezium task that moves mid-snapshot is still disruptive.

Tune rebalance behavior on the worker (connect-distributed.properties):

scheduled.rebalance.max.delay.ms=300000
connect.protocol=sessioned
session.timeout.ms=30000
heartbeat.interval.ms=10000

scheduled.rebalance.max.delay.ms (default 300000) lets the leader hold a departed worker’s tasks unassigned for up to five minutes, giving a restarting worker time to rejoin before its tasks are reassigned.

On the connector side, commit offsets often enough that a reassigned task resumes close to where it stopped:

{
  "offset.flush.interval.ms": "10000",
  "offset.flush.timeout.ms": "5000"
}

(offset.flush.interval.ms and offset.flush.timeout.ms are worker-level settings; set them in the worker properties rather than per connector if your platform requires it.) If a task is killed during a standard (blocking) snapshot, Debezium has no partial-snapshot offset and restarts the snapshot for its tables on reassignment — safe but wasteful. Incremental snapshots checkpoint per chunk and resume mid-snapshot, which is the real mitigation for snapshot interruption.

Troubleshooting Common Production Issues

Replication Slot Bloat (PostgreSQL)

A logical replication slot retains WAL until the subscriber confirms consumption. If the Debezium connector stops consuming — restart, network partition, or misconfiguration — the slot pins WAL and can exhaust disk.

Monitor slot lag:

SELECT slot_name,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS lag
FROM pg_replication_slots;

A heartbeat (heartbeat.interval.ms, shown earlier) is essential on low-traffic databases: without periodic writes, confirmed_flush_lsn never advances and the slot grows even when nothing relevant changed. This bites hardest when the captured tables are busy but the publication is narrow — the WAL fills with changes Debezium filters out, so it never acknowledges a newer LSN and the slot bloats despite an apparently healthy connector. If lag still climbs, fix the connector. As a last resort, drop and recreate the slot — which forces a fresh snapshot:

SELECT pg_drop_replication_slot('debezium_inventory');

Binlog Retention and Expiry (MySQL)

MySQL purges binary logs based on binlog_expire_logs_seconds (MySQL 8.0+) or the deprecated expire_logs_days (removed in MySQL 8.4). If the connector falls behind and a required binlog is purged, it fails with “Could not find first log file name in binary log index”, and recovery requires a fresh snapshot.

Set retention to at least seven days, persisting it across restarts:

SET PERSIST binlog_expire_logs_seconds = 604800;

SET PERSIST (MySQL 8.0+) writes to mysqld-auto.cnf so the value survives a restart; plain SET GLOBAL does not. On MySQL 5.7, use SET GLOBAL expire_logs_days = 7; and also set it in my.cnf.

Schema Registry Timeouts

When Debezium registers a new schema during a DDL change, the converter makes a synchronous HTTP call to the registry. If the registry is slow or unreachable, the task stalls. Bound that call:

{
  "key.converter.request.timeout.ms": "10000",
  "value.converter.request.timeout.ms": "10000",
  "key.converter.max.retries": "3",
  "value.converter.max.retries": "3"
}

(These pass through to the Confluent Avro converter / its registry client. Confirm property names against your converter version, as the registry-client property surface has changed over time.)

Tombstone Events and Compaction

With tombstones.on.delete=true (the default), Debezium emits a tombstone (null-value) record after each delete so log compaction can drop the key. Downstream consumers must handle null values explicitly. To keep both deletes-as-tombstones and time-based cleanup, use a combined policy:

kafka-configs --bootstrap-server kafka:9092 \
  --entity-type topics \
  --entity-name dbserver1.public.customers \
  --alter --add-config cleanup.policy=compact,delete

Performance Tuning for High-Throughput Deployments

Key connector parameters for throughput and latency:

{
  "max.batch.size": "4096",
  "max.queue.size": "16384",
  "max.queue.size.in.bytes": "104857600",
  "poll.interval.ms": "100",
  "snapshot.fetch.size": "20000",
  "incremental.snapshot.chunk.size": "4096",
  "event.processing.failure.handling.mode": "warn",
  "provide.transaction.metadata": "true"
}

Guidance:

• max.batch.size — maximum number of change events Debezium processes per batch. Larger batches improve throughput at some latency cost.
• max.queue.size / max.queue.size.in.bytes — capacity of the in-memory queue between the log reader and the producer, by record count and by bytes respectively. The byte limit guards against a few very large rows blowing up heap; 0 disables it. Keep max.queue.size comfortably larger than max.batch.size (the example keeps a 4:1 ratio) so the reader can stay ahead of the producer.
• poll.interval.ms — how long the connector waits before polling for new change events when the queue is empty. Lower reduces latency but raises CPU.
• snapshot.fetch.size — JDBC fetch size for the initial (blocking) snapshot’s row reads.
• incremental.snapshot.chunk.size — rows per chunk for incremental snapshots. Each chunk reads a bounded primary-key range, avoiding long transactions. Start near 4096 for tables over ~100 M rows and tune from observed chunk duration.
• event.processing.failure.handling.mode — fail (stop on a corrupt/unparseable event), warn (log and skip), or ignore (skip silently). There is no skip value.
• provide.transaction.metadata — when true, Debezium writes BEGIN/END boundary events to a separate transaction topic so consumers can group changes by source transaction.

Incremental Snapshots

Incremental snapshots (Debezium 1.6+) avoid the long read transaction of a blocking snapshot by chunking the table along its primary key and interleaving chunk reads with ongoing streaming. They are watermark-based and resumable: Debezium writes a low and high watermark around each chunk so concurrent streamed changes to the same rows are deduplicated against the READ events, and a task restart continues from the last completed chunk. They require a signal data collection:

{
  "signal.data.collection": "public.debezium_signal",
  "incremental.snapshot.chunk.size": "4096"
}

Create the signal table in the source database (the value must match signal.data.collection):

CREATE TABLE public.debezium_signal (
  id VARCHAR(42) PRIMARY KEY,
  type VARCHAR(32) NOT NULL,
  data VARCHAR(2048)
);

Trigger a snapshot at runtime by inserting an execute-snapshot signal — without restarting the connector:

INSERT INTO public.debezium_signal (id, type, data)
VALUES ('ad-hoc-1', 'execute-snapshot',
        '{"data-collections": ["public.orders"], "type": "incremental"}');

This re-snapshots a specific table on demand, which is the standard way to backfill a newly added table. (Debezium can also receive signals over a Kafka topic or JMX if you prefer not to write to the source database.)

Two requirements catch teams off guard. First, the chunking query orders by primary key, so the table must have a primary key; if it has only a multi-column key or none, supply a single unique column as a surrogate-key in the signal payload (Debezium 2.2+) or chunking degrades or fails. Second, you can scope a backfill with additional-conditions to copy only the rows that matter — useful when re-seeding one tenant or a recent date range rather than an entire history:

INSERT INTO public.debezium_signal (id, type, data)
VALUES ('backfill-2026', 'execute-snapshot',
        '{"data-collections": ["public.orders"], "type": "incremental",
          "additional-conditions": [
            {"data-collection": "public.orders", "filter": "created_at >= ''2026-01-01''"}
          ]}');

Sharding Across Multiple Connectors

At extreme scale, deploy multiple Debezium connectors against the same database with non-overlapping table filters, spreading the capture work across tasks and workers. For PostgreSQL, give each connector its own replication slot and publication; otherwise they contend for the same slot. See Kafka Connect Architecture and Deployment Modes for worker sizing.

Security and Authentication

Debezium connectors authenticate to both the source database and the Kafka cluster. Keep secrets out of connector JSON by using Connect’s ConfigProvider mechanism — FileConfigProvider (shown here) or a vault-backed provider — configured in the worker properties.

For PostgreSQL credentials sourced from files:

{
  "database.user": "${file:/etc/connector-secrets/pg.properties:username}",
  "database.password": "${file:/etc/connector-secrets/pg.properties:password}"
}

For Kafka cluster authentication, set producer overrides on the connector (producer.override.* is required when the worker enforces per-connector client overrides via connector.client.config.override.policy=All):

{
  "producer.override.security.protocol": "SASL_SSL",
  "producer.override.sasl.mechanism": "SCRAM-SHA-512",
  "producer.override.sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"debezium\" password=\"${file:/etc/connector-secrets/kafka.properties:password}\";"
}

Enable TLS on both legs. For PostgreSQL, set database.sslmode=verify-full (or at least require) and supply the CA/cert paths via the database.sslrootcert/database.sslcert/database.sslkey properties. For Kafka, configure truststore and keystore locations in the worker properties.

Grant the database user the minimum it needs: for PostgreSQL, REPLICATION plus SELECT on the captured tables and the schema-history requirements; the user does not need superuser, and avoiding it limits blast radius if the credential leaks.

Conclusion

Operating Debezium reliably comes down to understanding its internals rather than its happy path: snapshot mechanics, schema history versus logical-decoding metadata, rebalance behavior, and the real metric beans. The recurring failure modes are few and predictable — a pinned PostgreSQL slot or purged MySQL binlog forcing a re-snapshot, a breaking DDL change rejected by the registry, a snapshot restarting from zero after a task move — and each has a concrete mitigation above: heartbeats and slot monitoring, generous binlog retention, compatibility-aware schema changes, and incremental snapshots.

Pair these with the compatibility rules in Schema Evolution and Compatibility in Confluent Schema Registry and the deployment strategies in Kafka Connect Architecture and Deployment Modes to build CDC pipelines that hold up under critical workloads.

For further reference:

• Debezium documentation — connector-specific properties and behavior.
• Apache Kafka Connect documentation — Connect framework internals.
• Confluent Schema Registry documentation — compatibility types and schema management.