You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2014/02/18 13:23:12 UTC
Review Request 18211: Actual configs not updated after restart of host
component
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/
-----------------------------------------------------------
Review request for Ambari and Sid Wagle.
Bugs: AMBARI-4708
https://issues.apache.org/jira/browse/AMBARI-4708
Repository: ambari
Description
-------
Updated HDFS configs aren't applied after RESTART.
Steps:
- On a 3 node cluster, create a ConfigGroup for datanode.
- Override heap size to 1025m instead of 1024m
- Restart DN.
Result:
- The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
{code}
"global": {"2": "version1392171779044", "tag": "version1"}
{code}
- The API call still shows global as default version:
{code}
http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
{code}
Note:
It works after agent is restarted.
{code}
{
global: {
overrides: {
2: "version1392171779044"
},
default: "version1"
}
{code}
The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
Diffs
-----
ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
Diff: https://reviews.apache.org/r/18211/diff/
Testing
-------
1 Unit test failed, as it was before applying the patch.
Tests in error:
testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
Thanks,
Dmytro Sen
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/#review35033
-----------------------------------------------------------
Ship it!
Ship It!
- Sid Wagle
On Feb. 19, 2014, 6:14 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18211/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 6:14 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-4708
> https://issues.apache.org/jira/browse/AMBARI-4708
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Updated HDFS configs aren't applied after RESTART.
>
> Steps:
>
> - On a 3 node cluster, create a ConfigGroup for datanode.
> - Override heap size to 1025m instead of 1024m
> - Restart DN.
>
> Result:
> - The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
> {code}
> "global": {"2": "version1392171779044", "tag": "version1"}
> {code}
> - The API call still shows global as default version:
> {code}
> http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
> {code}
>
> Note:
> It works after agent is restarted.
>
> {code}
> {
> global: {
> overrides: {
> 2: "version1392171779044"
> },
> default: "version1"
> }
> {code}
>
> The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
>
> Diff: https://reviews.apache.org/r/18211/diff/
>
>
> Testing
> -------
>
> 1 Unit test failed, as it was before applying the patch.
>
> Tests in error:
> testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
>
> Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Dmytro Sen <ds...@hortonworks.com>.
> On Feb. 19, 2014, 7:07 p.m., Nate Cole wrote:
> > Is there more code changes that just restart hooks? I see at least 3 files changed for rca-type data to the agent.
>
> Nate Cole wrote:
> * than
No, only changes related to restart hooks.
"rca_enabled" and "db_name properties" aren't sent to the agents with custom_commands like RESTART, changes on the server side fix this.
- Dmytro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/#review34900
-----------------------------------------------------------
On Feb. 19, 2014, 6:14 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18211/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 6:14 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-4708
> https://issues.apache.org/jira/browse/AMBARI-4708
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Updated HDFS configs aren't applied after RESTART.
>
> Steps:
>
> - On a 3 node cluster, create a ConfigGroup for datanode.
> - Override heap size to 1025m instead of 1024m
> - Restart DN.
>
> Result:
> - The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
> {code}
> "global": {"2": "version1392171779044", "tag": "version1"}
> {code}
> - The API call still shows global as default version:
> {code}
> http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
> {code}
>
> Note:
> It works after agent is restarted.
>
> {code}
> {
> global: {
> overrides: {
> 2: "version1392171779044"
> },
> default: "version1"
> }
> {code}
>
> The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
>
> Diff: https://reviews.apache.org/r/18211/diff/
>
>
> Testing
> -------
>
> 1 Unit test failed, as it was before applying the patch.
>
> Tests in error:
> testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
>
> Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Nate Cole <nc...@hortonworks.com>.
> On Feb. 19, 2014, 2:07 p.m., Nate Cole wrote:
> > Is there more code changes that just restart hooks? I see at least 3 files changed for rca-type data to the agent.
* than
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/#review34900
-----------------------------------------------------------
On Feb. 19, 2014, 1:14 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18211/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 1:14 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-4708
> https://issues.apache.org/jira/browse/AMBARI-4708
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Updated HDFS configs aren't applied after RESTART.
>
> Steps:
>
> - On a 3 node cluster, create a ConfigGroup for datanode.
> - Override heap size to 1025m instead of 1024m
> - Restart DN.
>
> Result:
> - The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
> {code}
> "global": {"2": "version1392171779044", "tag": "version1"}
> {code}
> - The API call still shows global as default version:
> {code}
> http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
> {code}
>
> Note:
> It works after agent is restarted.
>
> {code}
> {
> global: {
> overrides: {
> 2: "version1392171779044"
> },
> default: "version1"
> }
> {code}
>
> The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
>
> Diff: https://reviews.apache.org/r/18211/diff/
>
>
> Testing
> -------
>
> 1 Unit test failed, as it was before applying the patch.
>
> Tests in error:
> testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
>
> Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/#review34900
-----------------------------------------------------------
Is there more code changes that just restart hooks? I see at least 3 files changed for rca-type data to the agent.
- Nate Cole
On Feb. 19, 2014, 1:14 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18211/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 1:14 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-4708
> https://issues.apache.org/jira/browse/AMBARI-4708
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Updated HDFS configs aren't applied after RESTART.
>
> Steps:
>
> - On a 3 node cluster, create a ConfigGroup for datanode.
> - Override heap size to 1025m instead of 1024m
> - Restart DN.
>
> Result:
> - The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
> {code}
> "global": {"2": "version1392171779044", "tag": "version1"}
> {code}
> - The API call still shows global as default version:
> {code}
> http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
> {code}
>
> Note:
> It works after agent is restarted.
>
> {code}
> {
> global: {
> overrides: {
> 2: "version1392171779044"
> },
> default: "version1"
> }
> {code}
>
> The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
>
> Diff: https://reviews.apache.org/r/18211/diff/
>
>
> Testing
> -------
>
> 1 Unit test failed, as it was before applying the patch.
>
> Tests in error:
> testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
>
> Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/#review35000
-----------------------------------------------------------
Ship it!
Ah, okay - that makes sense. Thanks!
- Nate Cole
On Feb. 19, 2014, 1:14 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18211/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 1:14 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-4708
> https://issues.apache.org/jira/browse/AMBARI-4708
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Updated HDFS configs aren't applied after RESTART.
>
> Steps:
>
> - On a 3 node cluster, create a ConfigGroup for datanode.
> - Override heap size to 1025m instead of 1024m
> - Restart DN.
>
> Result:
> - The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
> {code}
> "global": {"2": "version1392171779044", "tag": "version1"}
> {code}
> - The API call still shows global as default version:
> {code}
> http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
> {code}
>
> Note:
> It works after agent is restarted.
>
> {code}
> {
> global: {
> overrides: {
> 2: "version1392171779044"
> },
> default: "version1"
> }
> {code}
>
> The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
>
> Diff: https://reviews.apache.org/r/18211/diff/
>
>
> Testing
> -------
>
> 1 Unit test failed, as it was before applying the patch.
>
> Tests in error:
> testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
>
> Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
>
>
> Thanks,
>
> Dmytro Sen
>
>
Re: Review Request 18211: Actual configs not updated after restart of host
component
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18211/
-----------------------------------------------------------
(Updated Feb. 19, 2014, 6:14 p.m.)
Review request for Ambari, Nate Cole and Sid Wagle.
Bugs: AMBARI-4708
https://issues.apache.org/jira/browse/AMBARI-4708
Repository: ambari
Description
-------
Updated HDFS configs aren't applied after RESTART.
Steps:
- On a 3 node cluster, create a ConfigGroup for datanode.
- Override heap size to 1025m instead of 1024m
- Restart DN.
Result:
- The /var/lib/ambari-agent/data/config.json, has the correct values for the config type:
{code}
"global": {"2": "version1392171779044", "tag": "version1"}
{code}
- The API call still shows global as default version:
{code}
http://hostname1:8080/api/v1/clusters/c1/hosts/hostname1/host_components/DATANODE
{code}
Note:
It works after agent is restarted.
{code}
{
global: {
overrides: {
2: "version1392171779044"
},
default: "version1"
}
{code}
The cause is that hooks aren't executed for the custom_command like RESTART. Since configs for HDFS are generated in hook.py, we must execute hook.py before custom commands or move config generation from hook.py.
Diffs
-----
ambari-agent/src/main/python/resource_management/libraries/script/hook.py c5cae63
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 7f2d1fb
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java a6e52b0
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java d607eb4
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 0b62c87
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 3239c7a
Diff: https://reviews.apache.org/r/18211/diff/
Testing
-------
1 Unit test failed, as it was before applying the patch.
Tests in error:
testDeleteUsers(org.apache.ambari.server.controller.AmbariManagementControllerTest): Could not remove user user1. System should have at least one user with administrator role.
Tests run: 1401, Failures: 0, Errors: 1, Skipped: 7
Thanks,
Dmytro Sen