You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2015/04/01 20:23:50 UTC
Review Request 32734: Stack-advisor mechanism when recommending
configuration-dependencies has incorrect services.json content
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32734/
-----------------------------------------------------------
Review request for Ambari, Myroslav Papirkovskyy and Srimanth Gunturi.
Bugs: AMBARI-10316
https://issues.apache.org/jira/browse/AMBARI-10316
Repository: ambari
Description
-------
When the POST call is made for recommending configuration-dependencies, we generate the services.json file with following extra content:
"changed-configurations" : [ {
"type" : "yarn-site",
"name" : "yarn.scheduler.maximum-allocation-mb"
} ],
"depended-configurations" : [ {
"type" : "mapred-site",
"name" : "yarn.app.mapreduce.am.resource.mb"
}, {
"type" : "yarn-site",
"name" : "yarn.scheduler.maximum-allocation-mb"
}, {
"type" : "mapred-site",
"name" : "yarn.app.mapreduce.am.admin-command-opts"
}, {
"type" : "mapred-site",
"name" : "mapreduce.map.memory.mb"
}, {
"type" : "mapred-site",
"name" : "yarn.app.mapreduce.am.command-opts"
}, {
"type" : "mapred-site",
"name" : "mapreduce.reduce.java.opts"
}, {
"type" : "mapred-site",
"name" : "mapreduce.reduce.memory.mb"
}, {
"type" : "mapred-site",
"name" : "mapreduce.task.io.sort.mb"
}, {
"type" : "mapred-site",
"name" : "mapreduce.map.java.opts"
} ]
The configuration depended-by should not be added on to the services.json like this. All the configuration dependencies should be got from manipulating the org.apache.ambari.server.api.services.stackadvisor.commands.StackAdvisorCommand.GET_SERVICES_INFO_URI value. Changing the value to include configuration depends_on and depended_by information. Then the python script should build the dependency-chain inside itself instead of doing it in Java and adding to services.json.
The reason this is important is that for future functionality, we need services.json and hosts.json to faithfully represent the cluster's services and hosts information. The custom stack-advisor scripts should have logic to interpret it for their own needs. If we do some processing in Java, then stack-providers will not have chance to make their own interpretations.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 3a7fb6a
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java a7646a0
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackLevelConfigurationResourceProvider.java f010b37
ambari-server/src/main/resources/scripts/stack_advisor.py 1d67872
ambari-server/src/main/resources/stacks/stack_advisor.py c19a7ce
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java 22bece8
ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 76ecc96
Diff: https://reviews.apache.org/r/32734/diff/
Testing
-------
unt tests passed,
Thanks,
Dmytro Sen
Re: Review Request 32734: Stack-advisor mechanism when recommending
configuration-dependencies has incorrect services.json content
Posted by Srimanth Gunturi <sr...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32734/#review78587
-----------------------------------------------------------
Ship it!
Ship It!
- Srimanth Gunturi
On April 1, 2015, 6:23 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32734/
> -----------------------------------------------------------
>
> (Updated April 1, 2015, 6:23 p.m.)
>
>
> Review request for Ambari, Myroslav Papirkovskyy and Srimanth Gunturi.
>
>
> Bugs: AMBARI-10316
> https://issues.apache.org/jira/browse/AMBARI-10316
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When the POST call is made for recommending configuration-dependencies, we generate the services.json file with following extra content:
> "changed-configurations" : [ {
> "type" : "yarn-site",
> "name" : "yarn.scheduler.maximum-allocation-mb"
> } ],
> "depended-configurations" : [ {
> "type" : "mapred-site",
> "name" : "yarn.app.mapreduce.am.resource.mb"
> }, {
> "type" : "yarn-site",
> "name" : "yarn.scheduler.maximum-allocation-mb"
> }, {
> "type" : "mapred-site",
> "name" : "yarn.app.mapreduce.am.admin-command-opts"
> }, {
> "type" : "mapred-site",
> "name" : "mapreduce.map.memory.mb"
> }, {
> "type" : "mapred-site",
> "name" : "yarn.app.mapreduce.am.command-opts"
> }, {
> "type" : "mapred-site",
> "name" : "mapreduce.reduce.java.opts"
> }, {
> "type" : "mapred-site",
> "name" : "mapreduce.reduce.memory.mb"
> }, {
> "type" : "mapred-site",
> "name" : "mapreduce.task.io.sort.mb"
> }, {
> "type" : "mapred-site",
> "name" : "mapreduce.map.java.opts"
> } ]
> The configuration depended-by should not be added on to the services.json like this. All the configuration dependencies should be got from manipulating the org.apache.ambari.server.api.services.stackadvisor.commands.StackAdvisorCommand.GET_SERVICES_INFO_URI value. Changing the value to include configuration depends_on and depended_by information. Then the python script should build the dependency-chain inside itself instead of doing it in Java and adding to services.json.
> The reason this is important is that for future functionality, we need services.json and hosts.json to faithfully represent the cluster's services and hosts information. The custom stack-advisor scripts should have logic to interpret it for their own needs. If we do some processing in Java, then stack-providers will not have chance to make their own interpretations.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 3a7fb6a
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java a7646a0
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackLevelConfigurationResourceProvider.java f010b37
> ambari-server/src/main/resources/scripts/stack_advisor.py 1d67872
> ambari-server/src/main/resources/stacks/stack_advisor.py c19a7ce
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java 22bece8
> ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 76ecc96
>
> Diff: https://reviews.apache.org/r/32734/diff/
>
>
> Testing
> -------
>
> unt tests passed,
>
>
> Thanks,
>
> Dmytro Sen
>
>