[jira] [Created] (KAFKA-14390) Kafka and Zookeeper with FIPS SASL has error

["lkgen (Jira)" <ji...@apache.org>]

lkgen created KAFKA-14390:
-----------------------------

             Summary: Kafka and Zookeeper with FIPS SASL has error
                 Key: KAFKA-14390
                 URL: https://issues.apache.org/jira/browse/KAFKA-14390
             Project: Kafka
          Issue Type: Bug
          Components: security
    Affects Versions: 3.3.1, 2.7.1
            Reporter: lkgen


When setting Zookeeper to work with SASL plain and Kafka to connect to zookeeper with SASL and Java has FIPS enabled on a FIPS enabled Redhat machine
Kafka cannot connect with zookeeper

Used Apache Kafka 3.3.1 for both zookeeper and client
Operating system Redhat 8
Java 11 installed using yum install java-11-openjdk
set fips mode using command as root
fips-mode-setup --enable
rebooted machine using
systemctl reboot
verified fips mode with
fips-mode-setup --check

set JAVA_HOME to /usr/lib/jvm/jre-11

in config/zookeeper.properties, added properties
admin.enableServer = false
authProvider.1 = org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme = sasl

in config/server.properties, added properties
advertised.listeners = SASL_PLAINTEXT://localhost:9092
listeners = SASL_PLAINTEXT://:9092
security.inter.broker.protocol = SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol = PLAIN
sasl.enabled.mechanisms = PLAIN
super.users = User:admin

$HOME/zookeeper_jaas.conf value:
Server {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="0x572f372b5da34874a3fdf4d8002f"
    user_admin="0x572f372b5da34874a3fdf4d8002f";
};

$HOME/server_jaas.conf value:
KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="0x572f372b5da34874a3fdf4d8002f"
    user_admin="0x572f372b5da34874a3fdf4d8002f";
};
Client {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="0x572f372b5da34874a3fdf4d8002f";
};

for zookeeper, in kafka directory ran
setenv JAVA_HOME /usr/lib/jvm/jre-11
setenv KAFKA_OPTS -Djava.security.auth.login.config=$HOME/zookeeper_jaas.conf
./bin/zookeeper-server-start.sh ./config/zookeeper.properties

for kafka, in kafka directory ran
setenv JAVA_HOME /usr/lib/jvm/jre-11
setenv KAFKA_OPTS -Djava.security.auth.login.config=$HOME/server_jaas.conf
./bin/kafka-server-start.sh ./config/server.properties

Kafka exits with SASL error
[2022-11-15 13:21:04,344] ERROR SASL authentication with Zookeeper Quorum member failed. (org.apache.zookeeper.ClientCnxn)
javax.security.sasl.SaslException: saslClient failed to initialize properly: it's null.
        at org.apache.zookeeper.client.ZooKeeperSaslClient.initialize(ZooKeeperSaslClient.java:399)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1220)
[2022-11-15 13:21:04,348] INFO EventThread shut down for session: 0x1000575ab110000 (org.apache.zookeeper.ClientCnxn)
[2022-11-15 13:21:04,348] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2022-11-15 13:21:04,351] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2022-11-15 13:21:04,355] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientAuthFailedException: Auth failed either before or while waiting for connection
        at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:260)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:108)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1980)
        at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:503)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:203)
        at kafka.Kafka$.main(Kafka.scala:109)
        at kafka.Kafka.main(Kafka.scala)
[2022-11-15 13:21:04,356] INFO shutting down (kafka.server.KafkaServer)
[2022-11-15 13:21:04,363] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2022-11-15 13:21:04,363] INFO shut down completed (kafka.server.KafkaServer)
[2022-11-15 13:21:04,363] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
kafka.zookeeper.ZooKeeperClientAuthFailedException: Auth failed either before or while waiting for connection
        at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:260)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:108)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1980)
        at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:503)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:203)
        at kafka.Kafka$.main(Kafka.scala:109)
        at kafka.Kafka.main(Kafka.scala)
[2022-11-15 13:21:04,368] INFO shutting down (kafka.server.KafkaServer)

When removing FIPS by changing the java conf/security/java.security to have
security.useSystemPropertiesFile=false

The problem does not happen but this property disables FIPS



--
This message was sent by Atlassian Jira
(v8.20.10#820010)