Configuring SASL/SCRAM with Confluent Platform and Kubernetes

SASL/SCRAM provides password-based authentication for Kafka without transmitting credentials in plaintext or requiring per-client certificate infrastructure. This guide covers end-to-end configuration for Confluent Platform on Kubernetes: credential generation, broker configuration, client setup, and operational rotation.

A note on metadata storage up front, because it changes several procedures below. Through Confluent Platform 7.x you may still run ZooKeeper-backed clusters, where SCRAM credentials live in ZooKeeper. Apache Kafka 4.0 (March 2025) removed ZooKeeper entirely, and KRaft is now the only supported mode; there, SCRAM credentials are stored in the cluster metadata log as UserScramCredentialsRecord entries. The kafka-configs workflow below works identically against either backend as long as you target a broker with --bootstrap-server, which is what makes it the portable choice.

SASL/SCRAM in the Kafka Security Model

SCRAM (Salted Challenge Response Authentication Mechanism) authenticates clients via a challenge-response exchange using salted password hashes. Kafka supports two mechanisms: SCRAM-SHA-256 and SCRAM-SHA-512. The broker stores only salted hashes plus per-credential salt, iteration count, stored key, and server key — never the plaintext password.

The authentication flow follows RFC 5802:

1. Client sends its username and a client nonce.
2. Broker returns a combined nonce, the salt, and the iteration count.
3. Client derives a salted password and sends a client proof.
4. Broker verifies the proof against the stored key and returns a server signature; the client verifies the server in turn (mutual proof of password knowledge without sending the password).

SCRAM does not encrypt data in transit. You must pair it with TLS by using the SASL_SSL security protocol. SASL_PLAINTEXT sends the SCRAM exchange unencrypted; SCRAM resists replay of the password hash but offers no confidentiality, so only use SASL_PLAINTEXT on a network segment you already trust.

SCRAM is pragmatic for Kubernetes, where dynamic scaling makes per-pod certificate management costly. Credentials are managed online against running brokers, so adding users and rotating passwords needs no restart.

Prerequisites

• Confluent Platform deployed on Kubernetes (Helm chart or Confluent for Kubernetes operator).
• For KRaft clusters: at least one SCRAM super-user must already exist (bootstrapped at format time with kafka-storage format --add-scram) before brokers can authenticate to each other. For ZooKeeper-backed CP 7.x, ZooKeeper must be healthy.
• TLS enabled on any listener that will carry SCRAM traffic.
• kubectl access to the namespace and the ability to exec into broker pods.
• The kafka-configs tool, which ships inside the Confluent Platform broker image.

Verify your environment:

# Confirm broker pods are running
kubectl get pods -n confluent -l app.kubernetes.io/name=kafka

# Check which security protocol each listener uses
kubectl exec -it kafka-0 -n confluent -- \
  grep -E 'listener.security.protocol.map|^listeners=' /etc/kafka/kafka.properties

Generating and Storing SCRAM Credentials

Manual Creation with kafka-configs

Create a SCRAM-SHA-512 credential for user app-producer:

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --alter --add-config 'SCRAM-SHA-512=[iterations=8192,password=changeme]' \
  --entity-type users --entity-name app-producer

The broker computes the salted hash and persists it; the plaintext password is never stored. If you omit iterations, Kafka defaults to 4096 (the minimum). --command-config points at the admin client’s own SASL_SSL credentials — once authentication is enabled, the tool itself must authenticate to the broker.

Note: Use --bootstrap-server, not --zookeeper. The --zookeeper flag was deprecated for credential management in Kafka 2.2 and removed in Kafka 3.0; it does not exist in Kafka 4.0 / current Confluent Platform. --bootstrap-server works against both ZooKeeper-backed and KRaft clusters because the broker performs the write.

Operator-Based Credential Management

With Confluent for Kubernetes, store the password in a Kubernetes Secret and reference it from the relevant custom resource. The operator reconciles the credential into the cluster automatically:

apiVersion: v1
kind: Secret
metadata:
  name: app-producer-scram
  namespace: confluent
type: Opaque
stringData:
  password: changeme

The exact CRD and field used to reference the Secret depend on your operator version; consult the Confluent for Kubernetes documentation. Treat the CRD-driven path as the source of truth in operator-managed clusters so manual kafka-configs edits are not reverted on the next reconcile.

Verifying Credential Storage

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --describe --entity-type users --entity-name app-producer

The output lists the configured mechanism(s) and the stored parameters (iterations, salt, stored key, server key). The password is not recoverable from this output.

Configuring Kafka Brokers

Listener Configuration

Define a SASL_SSL listener for clients and a separate internal listener for inter-broker traffic. Example broker properties:

listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:9093
advertised.listeners=INTERNAL://kafka-0.kafka.confluent.svc.cluster.local:9092,EXTERNAL://kafka.example.com:9093
listener.security.protocol.map=INTERNAL:SASL_SSL,EXTERNAL:SASL_SSL
inter.broker.listener.name=INTERNAL

sasl.enabled.mechanisms=SCRAM-SHA-512
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512

listener.name.external.sasl.enabled.mechanisms=SCRAM-SHA-512
listener.name.external.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required;
listener.name.internal.sasl.enabled.mechanisms=SCRAM-SHA-512
listener.name.internal.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="kafka-broker" password="${broker-password}";

Two casing gotchas trip people up here: listener names appear uppercase in listeners and the protocol map (EXTERNAL), but the per-listener override keys lowercase them (listener.name.external...). The mechanism token inside the JAAS key is also lowercase and dash-separated (scram-sha-512), even though sasl.enabled.mechanisms uses the uppercase canonical form (SCRAM-SHA-512).

The client-facing listener’s sasl.jaas.config carries no username/password — clients supply those. The inter-broker listener’s JAAS config does need a username/password, because each broker is a SCRAM client of its peers.

For Helm deployments, set these under the chart’s broker config overrides (commonly configOverrides.server in Confluent for Kubernetes, or kafka.configOverrides in community charts). For Confluent for Kubernetes, set them in the Kafka resource under spec.configOverrides.

Inter-Broker Authentication

If inter-broker traffic uses SCRAM, create a dedicated broker user (e.g., kafka-broker) with kafka-configs and reference it in the internal listener’s JAAS config above. sasl.mechanism.inter.broker.protocol must match a mechanism enabled on the inter-broker listener. In KRaft clusters this user (or a controller super-user) must exist before the brokers start — hence the --add-scram bootstrap at format time.

Apply listener changes via a rolling restart of the Kafka StatefulSet. Confluent for Kubernetes performs the rolling restart automatically when the resource changes; for community Helm charts, run helm upgrade and let the StatefulSet roll. Credential additions and rotations (next sections) are online and need no restart.

Configuring Kubernetes Clients

Java Clients

The Java client does not expand environment variables inside sasl.jaas.config. There is no ${VAR} substitution at this layer, so you must read the secret in code and assemble the JAAS string yourself:

String password = System.getenv("KAFKA_SASL_PASSWORD");

Properties props = new Properties();
props.put("bootstrap.servers", "kafka.example.com:9093");
props.put("security.protocol", "SASL_SSL");
props.put("sasl.mechanism", "SCRAM-SHA-512");
props.put("sasl.jaas.config", String.format(
    "org.apache.kafka.common.security.scram.ScramLoginModule required " +
    "username=\"app-producer\" password=\"%s\";", password));
props.put("ssl.truststore.location", "/etc/kafka/secrets/truststore.jks");
props.put("ssl.truststore.password", "changeit");

Inject the password from a Kubernetes Secret as an environment variable:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-producer
spec:
  template:
    spec:
      containers:
      - name: producer
        env:
        - name: KAFKA_SASL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: app-producer-scram
              key: password

This keeps the secret out of the image and out of ConfigMaps. If you would rather not build the JAAS string by hand, implement a sasl.client.callback.handler.class that fetches the password from your secret store — that is the supported way to keep cleartext passwords off disk.

Non-Java Clients (librdkafka)

The Python confluent-kafka client (and other librdkafka bindings) does not use JAAS; set sasl.username and sasl.password directly:

import os
from confluent_kafka import Producer

conf = {
    'bootstrap.servers': 'kafka.example.com:9093',
    'security.protocol': 'SASL_SSL',
    'sasl.mechanism': 'SCRAM-SHA-512',
    'sasl.username': 'app-producer',
    'sasl.password': os.environ['KAFKA_SASL_PASSWORD'],
    'ssl.ca.location': '/etc/kafka/secrets/ca-cert.pem',
}

producer = Producer(conf)

Mount the CA certificate and inject the password from Secrets, exactly as for the Java client. Never embed credentials in the container image.

Credential Rotation

A user may hold multiple SCRAM credentials per mechanism, which makes zero-downtime rotation possible — but only with the right delete semantics, covered below.

Add the New Credential

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --alter --add-config 'SCRAM-SHA-512=[password=newpassword]' \
  --entity-type users --entity-name app-producer

Re-running --add-config for the same mechanism replaces that user’s SCRAM-SHA-512 credential with the new password rather than appending a second one. Both the old and new passwords are not simultaneously valid in the way naive rotation guides claim. Plan rotation around this:

1. Add the new credential (the command above). The user’s SCRAM-SHA-512 secret is now the new password.
2. Roll your client Deployments to the updated Secret value. Clients still holding the old password will fail authentication until they pick up the new one, so do this promptly.

To minimize the failure window, update the Kubernetes Secret first, let clients roll and pick it up, and run --add-config to switch the broker-side credential at the moment most clients already have the new value. There is no broker-side overlap period for a single mechanism.

If you need a true overlap window, run the two mechanisms in parallel: keep clients on SCRAM-SHA-256 with the old password while you stand up SCRAM-SHA-512 with the new one, migrate clients, then drop the old mechanism. That is the only way to have two simultaneously valid passwords for one user.

Remove a Credential

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --alter --delete-config 'SCRAM-SHA-512' \
  --entity-type users --entity-name app-producer

--delete-config 'SCRAM-SHA-512' removes the user’s SCRAM-SHA-512 credential entirely. Any client still using it will immediately fail authentication, so only run this once all clients have moved off that mechanism.

Operator-Based Rotation

Update the Kubernetes Secret with the new password and let Confluent for Kubernetes reconcile it. Because the operator drives the credential, do not also edit it with kafka-configs — the next reconcile would revert your change. Integrate with the External Secrets Operator or HashiCorp Vault to automate the Secret update step in a rotation pipeline.

Monitoring and Troubleshooting

Key JMX Metrics

Authentication metrics are reported per listener and per network processor under socket-server-metrics:

Each listener/networkProcessor pair emits its own series, so aggregate (sum) across processors for a per-listener total. There is also a failed-authentication-rate per-second variant if you prefer rates. Scrape these with the JMX Exporter (bundled with Confluent Platform) and a Prometheus PodMonitor/ServiceMonitor. A rising failed-authentication-total is your first signal of a bad-credential rollout or an expired rotation.

Common Errors

Dynamic Debug Logging

Raise the log level for the SCRAM/SASL packages on a single broker without a restart, using the broker-loggers entity type (KIP-412, Kafka 2.4+):

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --alter --add-config 'org.apache.kafka.common.security=DEBUG' \
  --entity-type broker-loggers --entity-name 0

The config key is the logger (package) name itself; the value is the level. --entity-name is the broker ID. The change is in-memory only and reverts on broker restart. To put the logger back, delete it (it reverts to the root level):

kubectl exec -it kafka-0 -n confluent -- \
  kafka-configs --bootstrap-server localhost:9092 \
  --command-config /etc/kafka/admin.properties \
  --alter --delete-config 'org.apache.kafka.common.security' \
  --entity-type broker-loggers --entity-name 0

Inspect current levels with --describe --entity-type broker-loggers --entity-name 0, then watch the broker logs for the SCRAM server callback handler resolving (or rejecting) each credential.

Related Pages

• Securing Kafka with mTLS and SASL/SCRAM
• Operations, Security & Observability