Automated Repartitioning with Kafka Cruise Control

Introduction

Running Apache Kafka at scale exposes a tension between the data distribution you design at topic-creation time and the operational reality of production. When you first define your Partitioning Strategies for High-Throughput Topics, you make assumptions about traffic patterns, key distributions, and broker capacity that rarely hold for long. Hotspots emerge as some partitions take disproportionate traffic. Brokers drift out of balance as topics are added or retired. Disk utilization skews as retention policies interact with uneven message rates. Correcting these imbalances by hand is slow, error-prone, and risky inside a maintenance window.

Cruise Control closes that gap by continuously sampling cluster metrics and generating partition-reassignment plans that hold brokers to operator-defined goals. It was developed at LinkedIn and is maintained as the open-source linkedin/cruise-control project. It is not part of the Apache Kafka distribution; you deploy it as a separate service alongside your cluster. You define goals for CPU, network throughput, disk usage, and replica distribution, and Cruise Control keeps the cluster within them instead of leaving you to react after an incident.

The value goes beyond load balancing. Automated repartitioning supports elastic scaling — adding or removing brokers triggers a rebalance with no hand-written reassignment JSON. It enforces rack-aware replica placement to preserve availability during a zone failure, and it decides from real metrics rather than operator intuition.

This work sits inside the broader Topics, Partitions & Data Modeling discipline: partition placement directly affects consumer parallelism, producer throughput, and recovery time. When Cruise Control moves a partition it must respect the ordering guarantees applications depend on, particularly where Keying Strategies for Ordered Message Processing are in use. Kafka preserves per-partition order across a reassignment, but a poorly throttled move can still degrade latency on a critical stream.

This article is an operational guide to deploying, configuring, and running Cruise Control in production. Every command, config key, and metric below is verified against the Cruise Control wiki and source; the project evolves, so confirm specifics against your installed version.

Cruise Control Architecture and Core Concepts

Cruise Control runs as a standalone Java process that connects to the Kafka cluster as a consumer and an AdminClient. It does not run inside the brokers, which keeps it isolated and lets you upgrade it independently of Kafka. Its work splits across four subsystems: the Load Monitor, the Analyzer, the Anomaly Detector, and the Executor.

Metrics Reporter and Sample Storage

The metrics pipeline is a hard prerequisite. Cruise Control does not scrape brokers directly. Instead, each broker runs the CruiseControlMetricsReporter plugin, which periodically publishes raw broker metrics to a Kafka topic (__CruiseControlMetrics by default). Cruise Control consumes that topic to build its workload model. You install the reporter in each broker’s server.properties:

# Broker server.properties — install the Cruise Control metrics reporter
metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter
cruise.control.metrics.topic.auto.create=true
cruise.control.metrics.topic.num.partitions=1
cruise.control.metrics.topic.replication.factor=3

This needs the cruise-control-metrics-reporter JAR on the broker classpath and a rolling broker restart. Cruise Control then derives partition- and broker-level samples and persists them to its sample-store topics, __KafkaCruiseControlPartitionMetricSamples and __KafkaCruiseControlModelTrainingSamples, so model state survives a Cruise Control restart. The sample-store replication factor is controlled by sample.store.topic.replication.factor (default 2); set it to 3 for production durability.

The Load Monitor and Workload Model

The Load Monitor turns raw samples into a structured workload model capturing per-partition resource use across four dimensions: CPU, network in, network out, and disk. It does not act on instantaneous values — it maintains rolling windows so transient spikes are smoothed and the model reflects sustained load. The window geometry is set with these properties (real defaults shown):

# cruisecontrol.properties — Load Monitor windowing
num.partition.metrics.windows=5
partition.metrics.window.ms=300000
num.broker.metrics.windows=20
broker.metrics.window.ms=300000
metric.sampling.interval.ms=120000

Partition load history spans num.partition.metrics.windows * partition.metrics.window.ms; with the defaults above that is 25 minutes of history sampled every 2 minutes. A window only counts once it has at least min.samples.per.partition.metrics.window samples (default 1); until enough valid windows exist, load and proposal requests fail with NotEnoughValidWindowsException. This is why a freshly started Cruise Control cannot rebalance immediately: with the defaults it needs roughly 25 minutes of clean sampling before it will model the cluster.

The Analyzer and Goal-Based Optimization

The Analyzer generates reassignment proposals. Cruise Control uses a goal-based framework: you define an ordered list of goals, each describing a desired property of the cluster, and the Analyzer satisfies them in priority order, never sacrificing a higher-priority goal to satisfy a lower one.

Goals fall into three families: rack awareness (replicas of a partition spread across failure domains), resource capacity goals (no broker exceeds its CPU, network, or disk limit), and distribution goals (even spread of replicas and leaders). Cruise Control ships a comprehensive set, and you can implement custom goals against its goal interface.

The search is heuristic, not globally optimal — exhaustive optimization is infeasible for large clusters. The Analyzer proposes candidate moves, scores each against the goal hierarchy, and selects a plan that satisfies the goals while limiting the number of movements.

The Anomaly Detector

The Anomaly Detector continuously watches for conditions that warrant action. It recognizes four anomaly types: broker failures, goal violations, metric anomalies, and disk failures (a topic-anomaly finder also exists in recent versions). A goal violation fires when the cluster drifts outside a configured goal — for example, when the load gap between the busiest and least-busy broker exceeds the goal’s tolerance. With self-healing enabled for that anomaly type, Cruise Control remediates automatically; otherwise it surfaces the violation for an operator to act on.

# Start Cruise Control — the start script takes one argument: the properties file
./kafka-cruise-control-start.sh config/cruisecontrol.properties

The Executor and Partition Reassignment

Once a proposal is approved — automatically or by an operator — the Executor carries it out through Kafka’s reassignment APIs, respecting configurable throttles and concurrency limits so data movement does not starve normal traffic. It moves partitions in batches with progress checks between them.

For each move, Kafka adds the destination as a new follower, waits for it to catch up, then removes the old replica; leadership migrates via preferred-leader election. Per-partition ordering is preserved throughout. The Executor tracks progress and exposes it through JMX and the REST API, so stalls are observable.

Deploying Cruise Control in Production

A production deployment must be reachable, secured, and sized for the cluster it manages.

Infrastructure and Sizing

Cruise Control is a standalone JVM service, run on dedicated hosts or as a container. Memory is the constraining resource, because the workload model and metric-window history live on the heap; the footprint scales with brokers, topics, and partitions. For a large cluster (on the order of 100 brokers and 100k partitions), start with several CPU cores and an 8–16 GB heap, then tune from observed heap usage and GC behavior. Wider metric windows (see Advanced Tuning) increase the footprint, because every additional window multiplies the per-partition history held in memory. The default REST/UI port is 9090.

Security Configuration

Cruise Control needs admin access to read metadata and execute reassignments. In a secured cluster, give it SSL/TLS or SASL credentials; it supports the standard Kafka security protocols (SASL_SSL, SASL_PLAINTEXT, SSL) and mechanisms (PLAIN, SCRAM-SHA-256/512, GSSAPI). Use a dedicated principal rather than sharing admin credentials.

With ACLs enabled, the Cruise Control principal needs cluster-level rights to alter partition assignments and describe configs, plus Describe on the topics it models. Grant the minimum required for your Kafka version and audit it.

# Security configuration in cruisecontrol.properties
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="cruise-control" \
  password="secure-password-here";
ssl.truststore.location=/etc/security/kafka.truststore.jks
ssl.truststore.password=truststore-password
ssl.keystore.location=/etc/security/cruise-control.keystore.jks
ssl.keystore.password=keystore-password
ssl.key.password=key-password

Running Redundant Instances

Cruise Control does not provide an active/passive leader-election clustering mode out of the box; there are no --ha.enabled or instance-coordination flags on the start script. A common pattern is to run a second instance configured identically against the same cluster as a warm standby, fronted by a load balancer or managed by your orchestration layer (for example, a Kubernetes Deployment that reschedules a single replica). The active instance owns execution. If you run two instances live, ensure only one can trigger executions: both reading the same sample-store topics is harmless, but concurrent reassignment requests are not coordinated between them and can submit conflicting moves to Kafka.

# Standby instance — same start script, same config, on another host
./kafka-cruise-control-start.sh config/cruisecontrol.properties

Integration with Monitoring and Alerting

Cruise Control exposes its internal state via JMX under the kafka.cruisecontrol domain. Scrape it with the JMX Prometheus exporter (jmx_exporter). Useful sensors:

• kafka.cruisecontrol:name=AnomalyDetector.balancedness-score — overall balance, 100 is fully balanced, dropping toward -1 as brokers/disks fail.
• kafka.cruisecontrol:name=AnomalyDetector.ongoing-anomaly-duration-ms — how long an unresolved anomaly has persisted.
• kafka.cruisecontrol:name=AnomalyDetector.number-of-self-healing-started — self-healing actions triggered.
• kafka.cruisecontrol:name=Executor.replica-action-in-progress and Executor.replica-action-pending — reassignment progress.

Alert on a balancedness score that stays depressed (a violation Cruise Control cannot fix) or on ongoing-anomaly-duration-ms growing without bound. The exact Prometheus metric name depends on your exporter’s jmx_exporter rules; the example below assumes a rule that maps the balancedness-score MBean to cruisecontrol_anomaly_balancedness_score. For a complete integration walkthrough see Automating Topic Repartitioning with Cruise Control and Prometheus Alerts.

# Prometheus alerting rule — name reflects your jmx_exporter mapping of the
# kafka.cruisecontrol:name=AnomalyDetector.balancedness-score MBean
groups:
  - name: cruise_control_alerts
    rules:
      - alert: CruiseControlClusterImbalanced
        expr: cruisecontrol_anomaly_balancedness_score < 80
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "Cruise Control balancedness score is low"
          description: "Balancedness score {{ $value }} has stayed below 80 for 15m."

Configuring Goals and Constraints

The value of automated repartitioning depends on how well the goal hierarchy and constraints match your workload. A poor hierarchy causes excessive movement, wasted network, or failure to address the real bottleneck.

The Goal Hierarchy

Goals are evaluated in strict priority order: the first goal is the most important, and Cruise Control will not violate it to satisfy a lower one. Putting RackAwareGoal first guarantees it never co-locates a partition’s replicas in one rack to improve CPU balance.

Cruise Control reads four overlapping goal lists from cruisecontrol.properties, and they nest as a strict subset chain — getting this nesting wrong is the most common reason self-healing silently never fires:

anomaly.detection.goals ⊆ self.healing.goals ⊆ default.goals ⊆ goals, with hard.goals a subset of every level.

• goals — the full set the Analyzer can use, in priority order.
• default.goals — the subset applied when a request names no goals; must be a subset of goals and a superset of hard.goals.
• hard.goals — goals that must never be violated; a proposal that breaks one is rejected outright.
• self.healing.goals — the goals used to build remediation proposals; must be a subset of default.goals.
• anomaly.detection.goals — the goals whose violation the detector reacts to; must be a subset of self.healing.goals, or the violation is detected but can never be auto-fixed.

A reasonable production ordering:

1. RackAwareGoal — spread replicas across racks for fault tolerance
2. ReplicaCapacityGoal — keep the replica count per broker under its limit
3. DiskCapacityGoal — keep disk usage per broker under capacity
4. NetworkInboundCapacityGoal / NetworkOutboundCapacityGoal — avoid network saturation
5. CpuCapacityGoal — keep CPU under its limit
6. ReplicaDistributionGoal — even replica count across brokers
7. LeaderReplicaDistributionGoal — even leader count across brokers
8. TopicReplicaDistributionGoal — spread each topic’s replicas evenly

Note the distinction worth getting right: ReplicaCapacityGoal bounds the number of replicas a broker may hold, while DiskCapacityGoal bounds disk bytes.

# Goal configuration in cruisecontrol.properties
goals=com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderReplicaDistributionGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal

hard.goals=com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal

Hard Goals vs. Soft Goals

Hard goals are absolute: a proposal that would violate one is discarded. Soft goals (everything in goals that is not a hard goal) are optimization targets Cruise Control tries to meet but may relax to satisfy a higher-priority goal. The rule of thumb:

Capacity Limits

Cruise Control reads broker capacities from a JSON file (capacity.config.file). Units matter and are not obvious: DISK is in MB, CPU is a percentage where 100 means all cores on the broker, and NW_IN / NW_OUT are in KB/s. The brokerId of -1 defines the default applied to every broker; an entry for a specific id overrides the default, which is how you model heterogeneous hardware.

{
  "brokerCapacities": [
    {
      "brokerId": "-1",
      "capacity": {
        "DISK": "1000000",
        "CPU": "100",
        "NW_IN": "100000",
        "NW_OUT": "100000"
      },
      "doc": "Default: DISK in MB, CPU as % (100 = all cores), NW in KB/s"
    },
    {
      "brokerId": "0",
      "capacity": {
        "DISK": "2000000",
        "CPU": "100",
        "NW_IN": "200000",
        "NW_OUT": "200000"
      },
      "doc": "Override for broker 0 with larger disk and network"
    }
  ]
}

Because CPU is a relative percentage, set capacity thresholds through goal settings rather than by understating the CPU value. Leave headroom in your network and disk numbers for replication traffic and maintenance — if allow_capacity_estimation is left on and the capacity file is missing a broker, Cruise Control will estimate its capacity and may plan moves against a wrong number, so prefer explicit entries for every distinct hardware class.

Movement Constraints and Throttling

Unrestricted reassignment can saturate inter-broker links. Cruise Control caps both throughput and concurrency:

# Execution throttling and concurrency (values from the shipped sample config)
default.replication.throttle=52428800
num.concurrent.partition.movements.per.broker=10
num.concurrent.leader.movements=1000
execution.progress.check.interval.ms=10000

default.replication.throttle is a replication byte-rate cap applied to the replicas being moved during execution (the example is 50 MB/s; the shipped value is empty, i.e. unthrottled — always set it). num.concurrent.partition.movements.per.broker bounds simultaneous replica moves into or out of a single broker (the sample config ships 10; the internal code default is 5). num.concurrent.leader.movements (default 1000) bounds leadership changes per batch, and execution.progress.check.interval.ms (default 10000) sets how often the Executor checks batch progress. Two cluster-wide ceilings cap total in-flight work: max.num.cluster.partition.movements (default 1250) for partition moves and max.num.cluster.movements (default 1250) for all movement types combined.

For clusters where a fixed throttle is too blunt, the concurrency adjuster can raise or lower movement concurrency dynamically based on cluster pressure. It is off by default and enabled per dimension: concurrency.adjuster.inter.broker.replica.enabled and concurrency.adjuster.leadership.enabled (both default false), optionally gated by concurrency.adjuster.min.isr.check.enabled (default false) so it backs off when partitions approach min-ISR. Start with a static throttle; reach for the adjuster only once you understand your cluster’s normal replication headroom.

Operational Workflows

Moving from manual to automated repartitioning is a phased transition: dry-run proposals and manual approval first, broader self-healing later. Cruise Control’s REST API is rooted at /kafkacruisecontrol/ on port 9090, and every mutating POST defaults to dryrun=true — you must pass dryrun=false to actually move data.

Generating and Reviewing Proposals

Request a rebalance proposal in dry-run mode to inspect what Cruise Control would do — number of movements, data to transfer, and resulting goal satisfaction — before committing:

# Dry-run rebalance proposal (default), filtered to the summary
curl -s -X POST "http://cruise-control:9090/kafkacruisecontrol/rebalance?dryrun=true&goals=RackAwareGoal,ReplicaCapacityGoal,CpuCapacityGoal,ReplicaDistributionGoal&allow_capacity_estimation=false&exclude_recently_demoted_brokers=true&exclude_recently_removed_brokers=true&verbose=true" \
  | jq '.summary'

Read the summary carefully, paying attention to partitions in latency-sensitive or strictly ordered topics. Setting allow_capacity_estimation=false forces Cruise Control to fail loudly rather than guess capacities when samples are missing — the safer default for a reviewed proposal.

Executing and Monitoring

To execute, re-issue the request with dryrun=false. Execution can run for hours on large clusters; watch both reassignment progress and cluster health.

Check execution state and stop an in-flight execution if conditions worsen:

# Inspect overall state, including the executor substate
curl -s "http://cruise-control:9090/kafkacruisecontrol/state?substates=executor" | jq '.'

# Stop the current execution (stops the *next* batch; the in-flight batch finishes)
curl -s -X POST "http://cruise-control:9090/kafkacruisecontrol/stop_proposal_execution"

There is no resume endpoint. stop_proposal_execution prevents the next batch from starting — the batch already submitted to Kafka completes — and the same is true if the Cruise Control process is bounced mid-execution. To finish an interrupted reassignment, re-issue the original rebalance, add_broker, or remove_broker request; Cruise Control will not start a competing execution while Kafka still has movements in flight.

Broker Failures and Cluster Expansion

When a broker fails, the Anomaly Detector raises a broker-failure anomaly; with broker-failure self-healing enabled, Cruise Control moves the affected replicas to healthy brokers automatically. Response time depends on the sampling/detection cadence and the failure-detection threshold.

For planned expansion, add the brokers to Kafka, then tell Cruise Control to fold them in with the dedicated add_broker endpoint (the generic rebalance endpoint does not take a brokerid parameter):

# Bring new brokers 5 and 6 into the balance and move data onto them
curl -s -X POST "http://cruise-control:9090/kafkacruisecontrol/add_broker?brokerid=5,6&dryrun=false&goals=RackAwareGoal,ReplicaCapacityGoal,CpuCapacityGoal,ReplicaDistributionGoal"

To decommission, use remove_broker?brokerid=..., which drains replicas off the listed brokers before they are removed.

Self-Healing

For autonomous operation, enable self-healing per anomaly type. The self-healing goal set is configured with self.healing.goals, and you opt in to specific anomaly types individually rather than all at once:

# Self-healing configuration
self.healing.enabled=true
self.healing.broker.failure.enabled=true
self.healing.goal.violation.enabled=false
self.healing.metric.anomaly.enabled=false
self.healing.disk.failure.enabled=true
self.healing.goals=com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,\
com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal
self.healing.exclude.recently.demoted.brokers=true
self.healing.exclude.recently.removed.brokers=true

Respect the subset chain from earlier: self.healing.goals must be a subset of default.goals, and anomaly.detection.goals must in turn be a subset of self.healing.goals — otherwise a detected violation has no proposal that can fix it. Roll out gradually: enable broker- and disk-failure healing first, validate, then consider goal-violation healing. Keep the exclude.recently.demoted/removed guards on so self-healing does not undo a deliberate operator action (for example, immediately re-leadering a broker you just demoted for maintenance).

Advanced Configuration and Tuning

Sampling Windows for Bursty or Diurnal Workloads

Model accuracy depends on the sampling geometry. The defaults suit stable workloads; clusters with strong daily cycles benefit from more, wider windows so the model spans a full traffic cycle. The trade-off is heap: history size is num.partition.metrics.windows * partition.metrics.window.ms.

# Wider history for diurnal patterns (~24h of partition history)
num.partition.metrics.windows=48
partition.metrics.window.ms=1800000
metric.sampling.interval.ms=60000

Topic Exclusion

Some topics are poor candidates for automatic movement — strict-latency topics, stream-processing state stores, or topics with very large partitions whose moves would dominate the replication budget. Exclude them with a regex; Cruise Control leaves matching topics in place during rebalances:

# Exclude topics from automatic partition movement
topics.excluded.from.partition.movement=^critical-.*|^state-store-.*

Topic prioritization by weight is not a configuration property in upstream Cruise Control; to bias a specific rebalance, name topics or goals explicitly on the REST request instead.

Broker Demotion for Maintenance

Before maintenance, demote a broker so leadership moves off it while its replicas stay put — useful for a rolling restart. Demotion makes the broker’s replicas the least-preferred leaders and runs a preferred-leader election:

# Move leadership off broker 3 ahead of maintenance
curl -s -X POST "http://cruise-control:9090/kafkacruisecontrol/demote_broker?brokerid=3&dryrun=false"

After maintenance, run a normal rebalance (or add_broker if it was fully drained) to restore balance; there is no separate “promote” endpoint.

Troubleshooting

Proposal Generation Failures

Proposals fail when hard goals cannot be met, goals conflict, or computation times out. The most common cause is genuine capacity exhaustion — no movement can satisfy a capacity goal on an overloaded cluster, so the fix is more brokers or less load. If the limits are merely too conservative, correct the capacity file. Inspect why with a verbose dry run:

curl -s -X POST "http://cruise-control:9090/kafkacruisecontrol/rebalance?dryrun=true&verbose=true" | jq '.'

A frequent early-life error is NotEnoughValidWindowsException: Cruise Control has not yet collected enough metric windows to model the cluster. How to tell which it is: query state?substates=monitor and read the number of valid/monitored windows — if it is climbing, just wait for num.partition.metrics.windows valid windows to accumulate; if it is stuck at zero, the metrics reporter is not installed or __CruiseControlMetrics is not being written, so check that topic’s end-offset is advancing.

Reassignment Stalls

Reassignments stall when a destination broker can’t keep up — network congestion, disk I/O saturation, or file-descriptor exhaustion. The Executor reports progress through state?substates=executor and the Executor.replica-action-pending JMX sensor; a stall looks like replica-action-pending holding flat while replica-action-in-progress does not advance across several execution.progress.check.interval.ms cycles. Identify the bottleneck from broker metrics and act: raise default.replication.throttle if the link is underused, or relieve the destination broker if it is saturated.

# Executor substate, including in-flight and pending actions
curl -s "http://cruise-control:9090/kafkacruisecontrol/state?substates=executor" | jq '.executorState'

Anomaly Detector False Positives

In bursty clusters, a batch job’s CPU spike can read as a goal violation. The Load Monitor’s windowing already smooths short spikes, so the first lever is window geometry — wider windows and more samples per window make the model less twitchy. Keep goal-violation self-healing disabled until you have characterized your cluster’s normal variance, and rely on the balancedness-score trend rather than instantaneous readings when deciding whether an imbalance is real.

State Recovery After a Cruise Control Restart

Cruise Control’s model state is rebuilt from the sample-store topics, so it recovers automatically after a restart. In-flight reassignments are owned by Kafka, not Cruise Control: a batch already submitted to the cluster completes on its own, but the next batch is not auto-started after a bounce. Re-issue the original rebalance/add_broker/remove_broker request to drive the remaining movements to completion. There is no execution.recovery.policy setting for this — it is inherent to how execution and Kafka reassignment interact.

Testing and Validating

Validate configuration in staging and roll out automation gradually before trusting it in production.

Staging Tests

A staging cluster mirroring production topology — even at reduced scale — lets you exercise goal hierarchies, capacity limits, and self-healing under control. Simulate broker failures, add and remove brokers, and inject load that trips goal violations, then confirm Cruise Control responds as intended. Verify reassignments do not cause unacceptable consumer latency, and that per-partition ordering holds for ordering-sensitive consumers.

# Simulate a broker failure in staging
systemctl stop kafka

# Give Cruise Control time to sample, detect, and (if enabled) react,
# then inspect the anomaly-detector substate
curl -s "http://cruise-control-staging:9090/kafkacruisecontrol/state?substates=anomaly_detector" | jq '.'

Note the endpoint: anomaly status is a substate of /state, queried with substates=anomaly_detector — there is no standalone /anomalies endpoint.

Gradual Rollout

Treat the rollout as discrete phases, each with an exit criterion before you advance:

Keep the ability to disable execution instantly at every phase, and make sure on-call is trained on Cruise Control operations before widening scope.

Baselines and Rack-Awareness Audits

Capture baselines — producer latency percentiles, consumer lag, end-to-end time — before enabling automation, and compare against them afterward to catch regressions caused by reassignments. The Apache Kafka operations documentation covers the cluster-side metrics that complement Cruise Control’s own.

Rack awareness is a hard guarantee worth verifying independently after large operations. The kafka_cluster_state endpoint exposes per-partition replica placement; audit it (or use kafka-topics.sh --describe) and alert on any partition whose replicas share a rack:

# Pull replica placement from Cruise Control for auditing
curl -s "http://cruise-control:9090/kafkacruisecontrol/kafka_cluster_state?json=true" | jq '.KafkaPartitionState'

Conclusion

Automated repartitioning with Cruise Control turns partition management from a manual, reactive chore into a monitored, goal-driven capability. The path runs through increasing operational maturity: first dry-run proposals reviewed and executed by hand, replacing ad-hoc reassignment scripts with constraint-aware plans; then automatic execution for well-understood cases like adding brokers or healing failures; finally broader self-healing backed by tuned anomaly detection and solid monitoring.

The payoff is less operational toil, steadier broker utilization, and faster response to topology changes. But Cruise Control amplifies good practice rather than replacing it: accurate capacity modeling, thoughtful topic design, and real monitoring still matter, and the judgment about what “balanced” means for your workload remains yours. Track upstream changes via the Kafka Improvement Proposals and the Cruise Control wiki to keep your configuration aligned with the version you run.