You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Shantanu Mundkur <an...@gmail.com> on 2016/03/18 23:31:01 UTC
Review Request 45056: Blueprint install using
config_recommendation_strategy is not functional
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/
-----------------------------------------------------------
Review request for Ambari, Oliver Szabo and Robert Levas.
Bugs: AMBARI-15454
https://issues.apache.org/jira/browse/AMBARI-15454
Repository: ambari
Description
-------
hBlueprint install using config_recommendation strategy seems to hang for a long time (couple of hours?) and ends up logging exceptions continually to ambari-server.log. At the same time many hundreds of directories are seen getting created under /var/run/ambari-server/stack-recommendations (I have seen above 800-900). If you keep it running eventually the cluster install seems to start but fails miserably at least during the start and some of it makes obvious that configuration recommendations were NOT applied. You see errors during startup that hint that options used for JVM (For Datanode etc) were unreasonable.
Note that both the blueprint and cluster templates used empty configurations. Example:
.
.
"configurations" : [],
"host_groups": [
{
"name": "host-group-1",
"configurations" : [],
"cardinality" : "1",
"components": [
{ "name": "APP_TIMELINE_SERVER" },
{ "name": "DATANODE" },
{ "name": "FALCON_CLIENT" },
{ "name": "FALCON_SERVER" },
{ "name": "FLUME_HANDLER" },
{ "name": "HBASE_CLIENT" },
{ "name": "HBASE_MASTER" },
{ "name": "HBASE_REGIONSERVER" },
.
.
.
Teh cluster template was:
{
"blueprint": "1node",
"config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
"default_password": "myPassword1",
"host_groups": [
{
"name": "host-group-1",
"hosts": [
{
"fqdn": "mynode.ibm.com"
}
]
}
]
}
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f5e7578
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d5755
Diff: https://reviews.apache.org/r/45056/diff/
Testing
-------
1) ambari-server unit tests
Results :
Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
2) Added unit test to ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY"
Thanks,
Shantanu Mundkur
Re: Review Request 45056: Blueprint install using
config_recommendation_strategy is not functional
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/#review125100
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On March 18, 2016, 6:31 p.m., Shantanu Mundkur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45056/
> -----------------------------------------------------------
>
> (Updated March 18, 2016, 6:31 p.m.)
>
>
> Review request for Ambari, Oliver Szabo and Robert Levas.
>
>
> Bugs: AMBARI-15454
> https://issues.apache.org/jira/browse/AMBARI-15454
>
>
> Repository: ambari
>
>
> Description
> -------
>
> hBlueprint install using config_recommendation strategy seems to hang for a long time (couple of hours?) and ends up logging exceptions continually to ambari-server.log. At the same time many hundreds of directories are seen getting created under /var/run/ambari-server/stack-recommendations (I have seen above 800-900). If you keep it running eventually the cluster install seems to start but fails miserably at least during the start and some of it makes obvious that configuration recommendations were NOT applied. You see errors during startup that hint that options used for JVM (For Datanode etc) were unreasonable.
>
> Note that both the blueprint and cluster templates used empty configurations. Example:
>
> .
> .
> "configurations" : [],
> "host_groups": [
> {
> "name": "host-group-1",
> "configurations" : [],
> "cardinality" : "1",
> "components": [
> { "name": "APP_TIMELINE_SERVER" },
> { "name": "DATANODE" },
> { "name": "FALCON_CLIENT" },
> { "name": "FALCON_SERVER" },
> { "name": "FLUME_HANDLER" },
> { "name": "HBASE_CLIENT" },
> { "name": "HBASE_MASTER" },
> { "name": "HBASE_REGIONSERVER" },
> .
> .
> .
>
> Teh cluster template was:
> {
> "blueprint": "1node",
> "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
> "default_password": "myPassword1",
> "host_groups": [
> {
> "name": "host-group-1",
> "hosts": [
> {
> "fqdn": "mynode.ibm.com"
> }
> ]
> }
> ]
> }
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f5e7578
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d5755
>
> Diff: https://reviews.apache.org/r/45056/diff/
>
>
> Testing
> -------
>
> 1) ambari-server unit tests
>
> Results :
>
> Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
> 2) Added unit test to ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY"
>
>
> Thanks,
>
> Shantanu Mundkur
>
>
Re: Review Request 45056: Blueprint install using
config_recommendation_strategy is not functional
Posted by Shantanu Mundkur <an...@gmail.com>.
> On March 19, 2016, 1:39 p.m., Oliver Szabo wrote:
> > Ship It!
> >
> > In the near future I will create a patch for fixing some issues with stack advisor blueprint support (e.g.: same kind of error can happen if hosts are not registered during stack advisor processing)
Thanks Oliver. Once Robert has reviewed the change I would request one of you to push the change into trunk as I do not have the privileges to do so. Can I assume it would get included into 2.4 when the branch is created? Thank you.
- Shantanu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/#review124404
-----------------------------------------------------------
On March 18, 2016, 10:31 p.m., Shantanu Mundkur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45056/
> -----------------------------------------------------------
>
> (Updated March 18, 2016, 10:31 p.m.)
>
>
> Review request for Ambari, Oliver Szabo and Robert Levas.
>
>
> Bugs: AMBARI-15454
> https://issues.apache.org/jira/browse/AMBARI-15454
>
>
> Repository: ambari
>
>
> Description
> -------
>
> hBlueprint install using config_recommendation strategy seems to hang for a long time (couple of hours?) and ends up logging exceptions continually to ambari-server.log. At the same time many hundreds of directories are seen getting created under /var/run/ambari-server/stack-recommendations (I have seen above 800-900). If you keep it running eventually the cluster install seems to start but fails miserably at least during the start and some of it makes obvious that configuration recommendations were NOT applied. You see errors during startup that hint that options used for JVM (For Datanode etc) were unreasonable.
>
> Note that both the blueprint and cluster templates used empty configurations. Example:
>
> .
> .
> "configurations" : [],
> "host_groups": [
> {
> "name": "host-group-1",
> "configurations" : [],
> "cardinality" : "1",
> "components": [
> { "name": "APP_TIMELINE_SERVER" },
> { "name": "DATANODE" },
> { "name": "FALCON_CLIENT" },
> { "name": "FALCON_SERVER" },
> { "name": "FLUME_HANDLER" },
> { "name": "HBASE_CLIENT" },
> { "name": "HBASE_MASTER" },
> { "name": "HBASE_REGIONSERVER" },
> .
> .
> .
>
> Teh cluster template was:
> {
> "blueprint": "1node",
> "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
> "default_password": "myPassword1",
> "host_groups": [
> {
> "name": "host-group-1",
> "hosts": [
> {
> "fqdn": "mynode.ibm.com"
> }
> ]
> }
> ]
> }
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f5e7578
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d5755
>
> Diff: https://reviews.apache.org/r/45056/diff/
>
>
> Testing
> -------
>
> 1) ambari-server unit tests
>
> Results :
>
> Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
> 2) Added unit test to ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY"
>
>
> Thanks,
>
> Shantanu Mundkur
>
>
Re: Review Request 45056: Blueprint install using
config_recommendation_strategy is not functional
Posted by Shantanu Mundkur <an...@gmail.com>.
> On March 19, 2016, 1:39 p.m., Oliver Szabo wrote:
> > Ship It!
> >
> > In the near future I will create a patch for fixing some issues with stack advisor blueprint support (e.g.: same kind of error can happen if hosts are not registered during stack advisor processing)
>
> Shantanu Mundkur wrote:
> Thanks Oliver. Once Robert has reviewed the change I would request one of you to push the change into trunk as I do not have the privileges to do so. Can I assume it would get included into 2.4 when the branch is created? Thank you.
Hello Robert,
Would you be able to review the change as well? I would also request you to push the change into trunk as I do not have the privileges to do so. I appreciate your time on this. Thank you.
- Shantanu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/#review124404
-----------------------------------------------------------
On March 18, 2016, 10:31 p.m., Shantanu Mundkur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45056/
> -----------------------------------------------------------
>
> (Updated March 18, 2016, 10:31 p.m.)
>
>
> Review request for Ambari, Oliver Szabo and Robert Levas.
>
>
> Bugs: AMBARI-15454
> https://issues.apache.org/jira/browse/AMBARI-15454
>
>
> Repository: ambari
>
>
> Description
> -------
>
> hBlueprint install using config_recommendation strategy seems to hang for a long time (couple of hours?) and ends up logging exceptions continually to ambari-server.log. At the same time many hundreds of directories are seen getting created under /var/run/ambari-server/stack-recommendations (I have seen above 800-900). If you keep it running eventually the cluster install seems to start but fails miserably at least during the start and some of it makes obvious that configuration recommendations were NOT applied. You see errors during startup that hint that options used for JVM (For Datanode etc) were unreasonable.
>
> Note that both the blueprint and cluster templates used empty configurations. Example:
>
> .
> .
> "configurations" : [],
> "host_groups": [
> {
> "name": "host-group-1",
> "configurations" : [],
> "cardinality" : "1",
> "components": [
> { "name": "APP_TIMELINE_SERVER" },
> { "name": "DATANODE" },
> { "name": "FALCON_CLIENT" },
> { "name": "FALCON_SERVER" },
> { "name": "FLUME_HANDLER" },
> { "name": "HBASE_CLIENT" },
> { "name": "HBASE_MASTER" },
> { "name": "HBASE_REGIONSERVER" },
> .
> .
> .
>
> Teh cluster template was:
> {
> "blueprint": "1node",
> "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
> "default_password": "myPassword1",
> "host_groups": [
> {
> "name": "host-group-1",
> "hosts": [
> {
> "fqdn": "mynode.ibm.com"
> }
> ]
> }
> ]
> }
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f5e7578
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d5755
>
> Diff: https://reviews.apache.org/r/45056/diff/
>
>
> Testing
> -------
>
> 1) ambari-server unit tests
>
> Results :
>
> Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
> 2) Added unit test to ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY"
>
>
> Thanks,
>
> Shantanu Mundkur
>
>
Re: Review Request 45056: Blueprint install using
config_recommendation_strategy is not functional
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45056/#review124404
-----------------------------------------------------------
Ship it!
Ship It!
In the near future I will create a patch for fixing some issues with stack advisor blueprint support (e.g.: same kind of error can happen if hosts are not registered during stack advisor processing)
- Oliver Szabo
On March 18, 2016, 10:31 p.m., Shantanu Mundkur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45056/
> -----------------------------------------------------------
>
> (Updated March 18, 2016, 10:31 p.m.)
>
>
> Review request for Ambari, Oliver Szabo and Robert Levas.
>
>
> Bugs: AMBARI-15454
> https://issues.apache.org/jira/browse/AMBARI-15454
>
>
> Repository: ambari
>
>
> Description
> -------
>
> hBlueprint install using config_recommendation strategy seems to hang for a long time (couple of hours?) and ends up logging exceptions continually to ambari-server.log. At the same time many hundreds of directories are seen getting created under /var/run/ambari-server/stack-recommendations (I have seen above 800-900). If you keep it running eventually the cluster install seems to start but fails miserably at least during the start and some of it makes obvious that configuration recommendations were NOT applied. You see errors during startup that hint that options used for JVM (For Datanode etc) were unreasonable.
>
> Note that both the blueprint and cluster templates used empty configurations. Example:
>
> .
> .
> "configurations" : [],
> "host_groups": [
> {
> "name": "host-group-1",
> "configurations" : [],
> "cardinality" : "1",
> "components": [
> { "name": "APP_TIMELINE_SERVER" },
> { "name": "DATANODE" },
> { "name": "FALCON_CLIENT" },
> { "name": "FALCON_SERVER" },
> { "name": "FLUME_HANDLER" },
> { "name": "HBASE_CLIENT" },
> { "name": "HBASE_MASTER" },
> { "name": "HBASE_REGIONSERVER" },
> .
> .
> .
>
> Teh cluster template was:
> {
> "blueprint": "1node",
> "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY",
> "default_password": "myPassword1",
> "host_groups": [
> {
> "name": "host-group-1",
> "hosts": [
> {
> "fqdn": "mynode.ibm.com"
> }
> ]
> }
> ]
> }
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java f5e7578
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 68d5755
>
> Diff: https://reviews.apache.org/r/45056/diff/
>
>
> Testing
> -------
>
> 1) ambari-server unit tests
>
> Results :
>
> Tests run: 3968, Failures: 0, Errors: 0, Skipped: 33
> 2) Added unit test to ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
> 3) Deployed clusters 1-5 nodes specifying "config_recommendation_strategy" : "ONLY_STACK_DEFAULTS_APPLY"
>
>
> Thanks,
>
> Shantanu Mundkur
>
>