Step-by-Step Guide to Migrating from ZooKeeper to KRaft

Migrating a production Apache Kafka cluster from ZooKeeper to KRaft (Kafka Raft) removes a separate distributed system from your stack, simplifies operations, and lifts the partition-count ceiling ZooKeeper imposes. Critically, the supported migration path (KIP-866) is an online, rolling migration — your cluster keeps serving produce and consume traffic throughout, with no full shutdown and no metadata “export/import” step. This guide gives the exact, ordered procedure for executing it on a live cluster, including controller provisioning, the dual-write bridge, broker cutover, finalization, and rollback.

This migration changes your cluster’s control plane, so review the broader Cluster Architecture & Provisioning context first, and confirm the move is right for you via KRaft vs ZooKeeper: Choosing the Right Metadata Management.

How the migration actually works

The migration does not shut the cluster down or dump and reload metadata. Instead, KRaft controllers are brought up in a special migration mode alongside your existing ZooKeeper ensemble. The controllers read the current metadata out of ZooKeeper, then the cluster enters dual-write mode: every metadata change is written to both the KRaft metadata log and ZooKeeper. You then roll the brokers from ZooKeeper mode to KRaft mode one at a time, and finally remove ZooKeeper. Because both copies stay in sync during the migration, you can roll back to ZooKeeper at any point until you finalize.

The phases, in order:

1. Get the cluster onto a migration-capable “bridge” release.
2. Provision KRaft controllers in migration mode.
3. Enable the migration trigger on the brokers (rolling restart) — metadata copies over, cluster enters dual-write.
4. Reconfigure the brokers as KRaft brokers (rolling restart).
5. Finalize by removing the ZooKeeper configuration from the controllers, then decommission ZooKeeper.

Prerequisites and pre-migration assessment

Version. ZooKeeper-to-KRaft migration shipped as early access in Apache Kafka 3.4.0 and became production-ready in 3.6.0; run a recent 3.x (or 4.x, where KRaft is the only mode and ZooKeeper support is removed — so you must migrate before upgrading to 4.0). Every broker must already be running the same bridge-release version, with inter.broker.protocol.version set to that version, before you begin.

Check the running version of each broker:

kafka-broker-api-versions.sh --bootstrap-server broker1:9092 | head -n 1

Inventory and dependencies. Record each broker’s broker.id, host, rack, and log directories. Identify anything that talks to ZooKeeper directly — monitoring that scrapes zookeeper.* MBeans, automation that manages the ensemble, and any tooling that writes ACL znodes — because those break once ZooKeeper is gone.

Controller quorum. KRaft controllers form a Raft quorum, so use an odd count: 3 (tolerates 1 failure) for most clusters, 5 (tolerates 2) across availability zones. For large clusters use dedicated controller nodes; plan for ~4 vCPUs, 16 GB RAM, and fast SSD/NVMe, with 100 GB provisioned for the metadata log and its snapshots.

Step 1 — Provision KRaft controllers in migration mode

Create a controller.properties for each new controller node. The key migration settings are zookeeper.metadata.migration.enable=true plus the existing zookeeper.connect, which together tell the controller to start in migration mode and read from ZooKeeper:

process.roles=controller
node.id=3000
controller.quorum.voters=3000@controller1:9093,3001@controller2:9093,3002@controller3:9093
controller.listener.names=CONTROLLER
listeners=CONTROLLER://:9093

# Migration mode: connect to the existing ZooKeeper ensemble
zookeeper.metadata.migration.enable=true
zookeeper.connect=zk1:2181,zk2:2181,zk3:2181
log.dirs=/var/lib/kafka/metadata

The KRaft cluster must keep the same cluster ID as the ZooKeeper-based cluster. Read it from ZooKeeper, then format each controller’s metadata log with it:

# Read the existing cluster ID from ZooKeeper
zookeeper-shell.sh zk1:2181 get /cluster/id

# Format each controller's storage with that cluster ID
kafka-storage.sh format \
  --cluster-id <CLUSTER_ID_FROM_ZK> \
  --config /etc/kafka/controller.properties

Start the controllers, then confirm the quorum has elected a leader:

kafka-server-start.sh -daemon /etc/kafka/controller.properties

kafka-metadata-quorum.sh --bootstrap-controller controller1:9093 describe --status

The output shows the LeaderId, HighWatermark, and the current voters — exactly one leader with the rest as followers. At this point the controllers are running but the brokers are still ZooKeeper-led; nothing has migrated yet.

Step 2 — Enable the migration trigger on the brokers

Add the migration trigger to every broker’s server.properties. You point the brokers at the controller quorum and turn on the migration flag, but keep zookeeper.connect — the broker still talks to ZooKeeper during this phase:

# existing settings stay (broker.id, listeners, zookeeper.connect, ...)
inter.broker.protocol.version=3.6
zookeeper.metadata.migration.enable=true
controller.quorum.voters=3000@controller1:9093,3001@controller2:9093,3002@controller3:9093
controller.listener.names=CONTROLLER
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT

Perform a rolling restart — one broker at a time, waiting for under-replicated partitions to return to zero before moving on:

# per broker, in turn:
kafka-server-stop.sh
kafka-server-start.sh -daemon /etc/kafka/server.properties

# wait until this returns no partitions before the next broker
kafka-topics.sh --bootstrap-server broker1:9092 --describe --under-replicated-partitions

Once the last broker has restarted with the trigger set, the active controller automatically reads all metadata from ZooKeeper into the KRaft metadata log, then the cluster enters dual-write mode. The cluster stays available the whole time.

Step 3 — Confirm dual-write mode

Verify the metadata copy completed before touching the brokers again. Watch the active controller’s log for the completion message, and check the migration state metric:

# In the active controller's log:
#   "Completed migration of metadata from ZooKeeper to KRaft"
grep -i "Completed migration" /var/log/kafka/controller.log

# ZkMigrationState exposed via JMX; in MIGRATION/dual-write it is value 2
kafka-metadata-quorum.sh --bootstrap-controller controller1:9093 describe --status

In dual-write mode every metadata change lands in both the KRaft log and ZooKeeper. This is the safe soak point: run here for as long as you need (commonly hours to a couple of weeks) to validate behavior, because rollback is still possible while the controllers remain connected to ZooKeeper.

Step 4 — Migrate the brokers to KRaft mode

Now reconfigure each broker as a pure KRaft broker. Replace the legacy broker.id with node.id (use the same number), set process.roles=broker, and remove the ZooKeeper and migration settings; keep the controller-quorum pointers:

process.roles=broker
node.id=1            # same value as the old broker.id
controller.quorum.voters=3000@controller1:9093,3001@controller2:9093,3002@controller3:9093
controller.listener.names=CONTROLLER
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT

# removed: broker.id, zookeeper.connect, zookeeper.metadata.migration.enable,
#          inter.broker.protocol.version

# if you use ACLs, switch to the KRaft authorizer:
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer

Roll the brokers one at a time again, waiting for each to rejoin and for under-replicated partitions to clear before continuing. After the last broker is in KRaft mode, the brokers no longer use ZooKeeper, but the controllers still do (they keep dual-writing as a safety net), so rollback remains available until Step 5.

Step 5 — Finalize (point of no return)

Remove the migration settings from each controller’s controller.properties:

# delete these two lines:
# zookeeper.metadata.migration.enable=true
# zookeeper.connect=zk1:2181,zk2:2181,zk3:2181

Roll the controllers one at a time. Once they restart without the ZooKeeper configuration, the cluster is fully KRaft, dual-write stops, and rollback is no longer possible. You can now shut the ZooKeeper ensemble down and remove its monitoring and automation.

Rollback

Until you complete Step 5, you can revert to ZooKeeper because both metadata stores are still in sync:

1. Revert any brokers already moved to KRaft (Step 4) back to their ZooKeeper-mode server.properties (restore broker.id and zookeeper.connect, remove process.roles) and roll them.
2. With all brokers back in ZooKeeper mode, stop the KRaft controllers.
3. The cluster runs on ZooKeeper exactly as before.

Once the controllers are finalized in Step 5 they are no longer connected to ZooKeeper, so this path is closed — which is why a dual-write soak period before finalizing is worthwhile.

Post-migration validation

Exercise the cluster end to end before declaring success:

• Topic operations — create, describe, alter, and delete a test topic; confirm partition assignments respect rack awareness.
• Produce/consume — run a real end-to-end test with your message sizes and, if you use transactions, verify exactly-once still holds.
• Consumer offsets — confirm existing groups resume from their committed positions (kafka-consumer-groups.sh --bootstrap-server broker1:9092 --describe --all-groups).
• ACLs and security — re-list ACLs and test each auth path.
• Controller failover — stop the active controller and confirm a new leader is elected quickly with no partition leadership loss.

Update monitoring to track KRaft-specific signals instead of ZooKeeper: kafka.controller:type=KafkaController,name=ActiveControllerCount (must be 1) and the raft-metrics group (current-leader, commit-index, log lag).

Common pitfalls

• Skipping the bridge release. The migration only works from a migration-capable version (3.6.0+ for production) with all brokers on the same version and inter.broker.protocol.version set. Jumping straight to 4.0 is not a migration path — 4.0 has no ZooKeeper mode, so you must migrate first.
• Reusing or omitting the cluster ID. The KRaft controllers must be formatted with the existing cluster ID read from /cluster/id in ZooKeeper; a fresh ID creates a separate cluster.
• Removing zookeeper.connect too early. Brokers keep it through Steps 2–3; only Steps 4 (brokers) and 5 (controllers) drop ZooKeeper.
• Finalizing before soaking. Once the controllers drop the migration config, rollback is gone. Validate thoroughly in dual-write mode first.

FAQ