You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ambari.apache.org by Aravindan Vijayan <av...@hortonworks.com> on 2015/10/30 01:21:22 UTC

Review Request 39790: Warn if Xmn size is oversubscribed in AMS HBase master and region server config, Kafka fails to start unless AMS is installed

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39790/
-----------------------------------------------------------

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.

Bugs: AMBARI-13642 and AMBARI-13643
https://issues.apache.org/jira/browse/AMBARI-13642
https://issues.apache.org/jira/browse/AMBARI-13643

Repository: ambari

Description
-------

AMBARI-13642 : Kafka fails to start unless AMS is installed

PROBLEM
When Kafka is installed on a cluster where AMS was never installed, the installation fails.

BUG
The kafka.metrics.reporter property had a default value pointing to a Reporter class provided by AMS.

FIX
Change default value of kafka.metrics.reporter to empty string. Change the property to the appropriate AMS Kafka reporter class dynamically though stack recommendations, if AMS is installed.
----------------------------------------------------------------------------
AMBARI-13643 : Warn if Xmn size is oversubscribed in AMS HBase master and region server config

PROBLEM/BUG
While setting the AMS configuration, the user can set any unbounded value for the Xmn settings. This might lead to undesirable outcomes from the JVM.

FIX
Provide validation of Xmn settings based on AMS mode.
If AMS is operating in embedded mode
regionserver_xmn ---- 12-20% of ( hbase_master_heapsize + hbase_regionserver_heapsize)
If AMS is operating in distributed distributed mode
hbase_master_xmn ---- 12 - 20% of hbase_master_heapsize
regionserver_xmn ---- 12 - 20% of hbase_regionserver_heapsize

Diffs
-----

ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 3a348f8
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml 67a2318
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 5779211
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py b74298c

Diff: https://reviews.apache.org/r/39790/diff/

Testing
-------

Manual Testing done.

Relevant unit tests pass.

Thanks,

Aravindan Vijayan

Re: Review Request 39790: Warn if Xmn size is oversubscribed in AMS HBase master and region server config, Kafka fails to start unless AMS is installed

Posted by Sid Wagle <sw...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39790/#review104503
-----------------------------------------------------------


Changes look, good can you please add some unit tests?

- Sid Wagle


On Oct. 30, 2015, 12:21 a.m., Aravindan Vijayan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39790/
> -----------------------------------------------------------
> 
> (Updated Oct. 30, 2015, 12:21 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13642 and AMBARI-13643
>     https://issues.apache.org/jira/browse/AMBARI-13642
>     https://issues.apache.org/jira/browse/AMBARI-13643
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> AMBARI-13642 : Kafka fails to start unless AMS is installed
> 
> PROBLEM
> When Kafka is installed on a cluster where AMS was never installed, the installation fails.
> 
> BUG
> The kafka.metrics.reporter property had a default value pointing to a Reporter class provided by AMS.
> 
> FIX
> Change default value of kafka.metrics.reporter to empty string. Change the property to the appropriate AMS Kafka reporter class dynamically though stack recommendations, if AMS is installed.
> ----------------------------------------------------------------------------
> AMBARI-13643 : Warn if Xmn size is oversubscribed in AMS HBase master and region server config
> 
> PROBLEM/BUG
> While setting the AMS configuration, the user can set any unbounded value for the Xmn settings. This might lead to undesirable outcomes from the JVM.
> 
> FIX
> Provide validation of Xmn settings based on AMS mode.
> If AMS is operating in embedded mode
> regionserver_xmn ---- 12-20% of ( hbase_master_heapsize + hbase_regionserver_heapsize)
> If AMS is operating in distributed distributed mode
> hbase_master_xmn ---- 12 - 20% of hbase_master_heapsize
> regionserver_xmn ---- 12 - 20% of hbase_regionserver_heapsize
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 3a348f8 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml 67a2318 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 5779211 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py b74298c 
> 
> Diff: https://reviews.apache.org/r/39790/diff/
> 
> 
> Testing
> -------
> 
> Manual Testing done.
> 
> Relevant unit tests pass.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 39790: Warn if Xmn size is oversubscribed in AMS HBase master and region server config, Kafka fails to start unless AMS is installed

Posted by Sid Wagle <sw...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39790/#review104611
-----------------------------------------------------------

Ship it!


Ship It!

- Sid Wagle


On Oct. 30, 2015, 8:15 p.m., Aravindan Vijayan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39790/
> -----------------------------------------------------------
> 
> (Updated Oct. 30, 2015, 8:15 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-13642 and AMBARI-13643
>     https://issues.apache.org/jira/browse/AMBARI-13642
>     https://issues.apache.org/jira/browse/AMBARI-13643
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> AMBARI-13642 : Kafka fails to start unless AMS is installed
> 
> PROBLEM
> When Kafka is installed on a cluster where AMS was never installed, the installation fails.
> 
> BUG
> The kafka.metrics.reporter property had a default value pointing to a Reporter class provided by AMS.
> 
> FIX
> Change default value of kafka.metrics.reporter to empty string. Change the property to the appropriate AMS Kafka reporter class dynamically though stack recommendations, if AMS is installed.
> ----------------------------------------------------------------------------
> AMBARI-13643 : Warn if Xmn size is oversubscribed in AMS HBase master and region server config
> 
> PROBLEM/BUG
> While setting the AMS configuration, the user can set any unbounded value for the Xmn settings. This might lead to undesirable outcomes from the JVM.
> 
> FIX
> Provide validation of Xmn settings based on AMS mode.
> If AMS is operating in embedded mode
> regionserver_xmn ---- 12-20% of ( hbase_master_heapsize + hbase_regionserver_heapsize)
> If AMS is operating in distributed distributed mode
> hbase_master_xmn ---- 12 - 20% of hbase_master_heapsize
> regionserver_xmn ---- 12 - 20% of hbase_regionserver_heapsize
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 3a348f8 
>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/configuration/kafka-broker.xml 67a2318 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 5779211 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 52bf494 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 6b582c9 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py e677982 
> 
> Diff: https://reviews.apache.org/r/39790/diff/
> 
> 
> Testing
> -------
> 
> Manual Testing done.
> 
> Relevant unit tests pass.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 39790: Warn if Xmn size is oversubscribed in AMS HBase master and region server config, Kafka fails to start unless AMS is installed

Posted by Aravindan Vijayan <av...@hortonworks.com>.

(Updated Oct. 30, 2015, 8:15 p.m.)

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.

Changes
-------

Added Unit Tests and fixed issues.

Bugs: AMBARI-13642 and AMBARI-13643
https://issues.apache.org/jira/browse/AMBARI-13642
https://issues.apache.org/jira/browse/AMBARI-13643

Repository: ambari

Description
-------

AMBARI-13642 : Kafka fails to start unless AMS is installed

PROBLEM
When Kafka is installed on a cluster where AMS was never installed, the installation fails.

BUG
The kafka.metrics.reporter property had a default value pointing to a Reporter class provided by AMS.

PROBLEM/BUG
While setting the AMS configuration, the user can set any unbounded value for the Xmn settings. This might lead to undesirable outcomes from the JVM.

Diffs (updated)
-----

Diff: https://reviews.apache.org/r/39790/diff/

Testing
-------

Manual Testing done.

Relevant unit tests pass.

Thanks,

Aravindan Vijayan