You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2015/11/10 15:37:43 UTC

Review Request 40137: After kerberization, Kafka brokers fail to start

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/
-----------------------------------------------------------

Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.


Bugs: AMBARI-13813
    https://issues.apache.org/jira/browse/AMBARI-13813


Repository: ambari


Description
-------

PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 

Diff: https://reviews.apache.org/r/40137/diff/


Testing
-------

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Andrew Onischuk <ao...@hortonworks.com>.


> On Nov. 10, 2015, 2:58 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java, line 904
> > <https://reviews.apache.org/r/40137/diff/1/?file=1121811#file1121811line904>
> >
> >     can we include previous params here
> >     KAFKA_KERBEROS_PARAMS="$KAFKA_KERBEROS_PARAMS {{kafka_kerberos_params}}"
> 
> Dmytro Sen wrote:
>     If previous configs present - we do nothing
>     
>     if (kafkaEnvContent != null && !kafkaEnvContent.contains("export KAFKA_KERBEROS_PARAMS")) {
> 
> Andrew Onischuk wrote:
>     oh I see it now.

Actually I don't understand why do we need this condition. What happens if user specifies his own kafka params, but needs this fix as well?


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105874
-----------------------------------------------------------


On Nov. 10, 2015, 2:37 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2015, 2:37 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Andrew Onischuk <ao...@hortonworks.com>.


> On Nov. 10, 2015, 2:58 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java, line 904
> > <https://reviews.apache.org/r/40137/diff/1/?file=1121811#file1121811line904>
> >
> >     can we include previous params here
> >     KAFKA_KERBEROS_PARAMS="$KAFKA_KERBEROS_PARAMS {{kafka_kerberos_params}}"
> 
> Dmytro Sen wrote:
>     If previous configs present - we do nothing
>     
>     if (kafkaEnvContent != null && !kafkaEnvContent.contains("export KAFKA_KERBEROS_PARAMS")) {

oh I see it now.


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105874
-----------------------------------------------------------


On Nov. 10, 2015, 2:37 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2015, 2:37 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Dmytro Sen <ds...@hortonworks.com>.


> On Ноя. 10, 2015, 2:58 п.п., Andrew Onischuk wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java, line 904
> > <https://reviews.apache.org/r/40137/diff/1/?file=1121811#file1121811line904>
> >
> >     can we include previous params here
> >     KAFKA_KERBEROS_PARAMS="$KAFKA_KERBEROS_PARAMS {{kafka_kerberos_params}}"

If previous configs present - we do nothing

if (kafkaEnvContent != null && !kafkaEnvContent.contains("export KAFKA_KERBEROS_PARAMS")) {


- Dmytro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105874
-----------------------------------------------------------


On Ноя. 10, 2015, 2:37 п.п., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Ноя. 10, 2015, 2:37 п.п.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Andrew Onischuk <ao...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105874
-----------------------------------------------------------



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java (line 904)
<https://reviews.apache.org/r/40137/#comment164558>

    can we include previous params here
    KAFKA_KERBEROS_PARAMS="$KAFKA_KERBEROS_PARAMS {{kafka_kerberos_params}}"


- Andrew Onischuk


On Nov. 10, 2015, 2:37 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2015, 2:37 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105896
-----------------------------------------------------------

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Лис. 10, 2015, 5:01 після полудня, Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Лис. 10, 2015, 5:01 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Andrew Onischuk <ao...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105895
-----------------------------------------------------------

Ship it!


Ship It!

- Andrew Onischuk


On Nov. 10, 2015, 5:01 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Nov. 10, 2015, 5:01 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Dmytro Sen <ds...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/
-----------------------------------------------------------

(Updated Ноя. 10, 2015, 5:01 п.п.)

Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.

Bugs: AMBARI-13813
https://issues.apache.org/jira/browse/AMBARI-13813

Repository: ambari

Description
-------

PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf"
ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.

Diffs (updated)
-----

ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc

Diff: https://reviews.apache.org/r/40137/diff/

Testing
-------

Unit tests passed

Thanks,

Dmytro Sen

Re: Review Request 40137: After kerberization, Kafka brokers fail to start

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40137/#review105873
-----------------------------------------------------------



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java (line 86)
<https://reviews.apache.org/r/40137/#comment164556>

    KAFKA_ENV -> KAFKA_ENV_CONFIG



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java (line 906)
<https://reviews.apache.org/r/40137/#comment164557>

    Please add expect(...) in UnitTests for this.


- Vitalyi Brodetskyi


On Лис. 10, 2015, 2:37 після полудня, Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40137/
> -----------------------------------------------------------
> 
> (Updated Лис. 10, 2015, 2:37 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-13813
>     https://issues.apache.org/jira/browse/AMBARI-13813
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: After Ambari upgrade and kerberization, Kafka broker does not come up. Following error is observed:
> FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
> java.lang.IllegalArgumentException: You must pass java.security.auth.login.config in secure mode.
> STEPS TO REPRODUCE: Upgrade Ambari to 2.1.2 on a cluster without kerberization and with Kafka brokers, then kerberize through Ambari.
> EXPECTED RESULT: Kafka broker comes up successfully, and ambari sets the correct parameters in kafka-env.sh:
> export KAFKA_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_jaas.conf" 
> ACTUAL RESULT: Kafka broker fails to come up and ambari-server set kafka-env incorrectly.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java c72809c 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java 61050cc 
> 
> Diff: https://reviews.apache.org/r/40137/diff/
> 
> 
> Testing
> -------
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>