You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Vitalyi Brodetskyi <vb...@hortonworks.com> on 2017/07/17 20:14:02 UTC

Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/
-----------------------------------------------------------

Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21504
    https://issues.apache.org/jira/browse/AMBARI-21504


Repository: ambari


Description
-------

Steps:

* Installed Ambari 2.2 with Slider and services it required
* Upgraded Ambari to 2.5.2.0-134
* Registered HDP 2.6.1.0 repo, installed packages
* Restarted services that needed restart
* Ran service checks
* Started upgrade

Result: _Restarting History Server_ step failed with 

{noformat:title=errors-91.txt}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
    self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
    copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
    mode=0555
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
    self.action_delayed("create")
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
    parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
    math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
  File "/usr/lib64/python2.7/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
{noformat}

{noformat:title=command-91.json}
"hostLevelParams": {
...
"not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
...
}
{noformat}

Failure seems to be due to {{null}} value in the list.


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
  ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 


Diff: https://reviews.apache.org/r/60921/diff/1/


Testing
-------

mvn clean test


Thanks,

Vitalyi Brodetskyi


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

> On Липень 18, 2017, 7:15 до полудня, Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
> > Lines 227 (patched)
> > <https://reviews.apache.org/r/60921/diff/1/?file=1777978#file1777978line227>
> >
> >     Use `equals()` for string comparison, not `==`

OMG, python is killing java skills. Thanks for review


- Vitalyi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180797
-----------------------------------------------------------


On Липень 17, 2017, 8:14 після полудня, Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated Липень 17, 2017, 8:14 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180797
-----------------------------------------------------------


Fix it, then Ship it!




`'immutable_paths': [u'/mr-history/done', u'/app-logs']` is fine with this change.


ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
Lines 227 (patched)
<https://reviews.apache.org/r/60921/#comment256051>

    Use `equals()` for string comparison, not `==`



ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
Lines 764 (patched)
<https://reviews.apache.org/r/60921/#comment256055>

    `1` is the expected value, so it should be passed as first arg to `assertEquals(long expected, long actual)`.  Otherwise the message will be the wrong, eg.: `expected:<2> but was:<1>`.



ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
Lines 765 (patched)
<https://reviews.apache.org/r/60921/#comment256052>

    Use `assertEquals`, provides better output in case of failure


- Attila Doroszlai


On July 17, 2017, 10:14 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated July 17, 2017, 10:14 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

> On Липень 17, 2017, 10:39 після полудня, Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
> > Lines 100 (patched)
> > <https://reviews.apache.org/r/60921/diff/1/?file=1777979#file1777979line100>
> >
> >     With the java change, will this error path be hit?

Java part will just remove invalid values from list. In python side i've added validation for hdfs_tmp_dir, i thought it will be good to infrom user that he has potential problem. Also, if hdfs_tmp_dir not correct ("null" or empty), history server start will fail.


- Vitalyi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180757
-----------------------------------------------------------


On Липень 17, 2017, 8:14 після полудня, Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated Липень 17, 2017, 8:14 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.

> On Липень 17, 2017, 10:39 після полудня, Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
> > Lines 228 (patched)
> > <https://reviews.apache.org/r/60921/diff/1/?file=1777978#file1777978line228>
> >
> >     Should we add a log here that some property is being removed as its empty/null.

Sure, will add logging here too.


- Vitalyi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180757
-----------------------------------------------------------


On Липень 17, 2017, 8:14 після полудня, Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated Липень 17, 2017, 8:14 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180757
-----------------------------------------------------------




ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
Lines 228 (patched)
<https://reviews.apache.org/r/60921/#comment255976>

    Should we add a log here that some property is being removed as its empty/null.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
Lines 100 (patched)
<https://reviews.apache.org/r/60921/#comment255977>

    With the java change, will this error path be hit?


- Sumit Mohanty


On July 17, 2017, 8:14 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated July 17, 2017, 8:14 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180805
-----------------------------------------------------------


Fix it, then Ship it!




Tested new patch, it still works fine.


ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
Lines 20 (patched)
<https://reviews.apache.org/r/60921/#comment256077>

    Please don't reorder imports unnecessarily.  (This will cause checkstyle error on trunk.)



ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java
Line 21 (original), 21 (patched)
<https://reviews.apache.org/r/60921/#comment256078>

    Please don't reorder imports unnecessarily.  (This will cause checkstyle error on trunk.)


- Attila Doroszlai


On July 18, 2017, 2:11 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated July 18, 2017, 2:11 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java dbfee41 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/2/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/
-----------------------------------------------------------

(Updated Липень 18, 2017, 1:50 після полудня)


Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21504
    https://issues.apache.org/jira/browse/AMBARI-21504


Repository: ambari


Description
-------

Steps:

* Installed Ambari 2.2 with Slider and services it required
* Upgraded Ambari to 2.5.2.0-134
* Registered HDP 2.6.1.0 repo, installed packages
* Restarted services that needed restart
* Ran service checks
* Started upgrade

Result: _Restarting History Server_ step failed with 

{noformat:title=errors-91.txt}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
    self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
    copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
    mode=0555
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
    self.action_delayed("create")
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
    parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
    math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
  File "/usr/lib64/python2.7/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
{noformat}

{noformat:title=command-91.json}
"hostLevelParams": {
...
"not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
...
}
{noformat}

Failure seems to be due to {{null}} value in the list.


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
  ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java dbfee41 
  ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 


Diff: https://reviews.apache.org/r/60921/diff/3/

Changes: https://reviews.apache.org/r/60921/diff/2-3/


Testing
-------

mvn clean test


Thanks,

Vitalyi Brodetskyi


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/
-----------------------------------------------------------

(Updated Липень 18, 2017, 12:11 після полудня)


Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-21504
    https://issues.apache.org/jira/browse/AMBARI-21504


Repository: ambari


Description
-------

Steps:

* Installed Ambari 2.2 with Slider and services it required
* Upgraded Ambari to 2.5.2.0-134
* Registered HDP 2.6.1.0 repo, installed packages
* Restarted services that needed restart
* Ran service checks
* Started upgrade

Result: _Restarting History Server_ step failed with 

{noformat:title=errors-91.txt}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
    self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
    copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
    mode=0555
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
    self.action_delayed("create")
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
    parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
    math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
  File "/usr/lib64/python2.7/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or buffer
{noformat}

{noformat:title=command-91.json}
"hostLevelParams": {
...
"not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
...
}
{noformat}

Failure seems to be due to {{null}} value in the list.


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
  ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
  ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerImplTest.java dbfee41 
  ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 


Diff: https://reviews.apache.org/r/60921/diff/2/

Changes: https://reviews.apache.org/r/60921/diff/1-2/


Testing
-------

mvn clean test


Thanks,

Vitalyi Brodetskyi


Re: Review Request 60921: Restart of MR2 History Server failed due to null in immutable_paths

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60921/#review180800
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
Lines 48 (patched)
<https://reviews.apache.org/r/60921/#comment256066>

    "Ambari WILL NOT update permissions for it."
    
    I think if property is not in immutable paths,
    
    ambari WILL update permissions for it


- Andrew Onischuk


On July 17, 2017, 8:14 p.m., Vitalyi Brodetskyi wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60921/
> -----------------------------------------------------------
> 
> (Updated July 17, 2017, 8:14 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21504
>     https://issues.apache.org/jira/browse/AMBARI-21504
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Steps:
> 
> * Installed Ambari 2.2 with Slider and services it required
> * Upgraded Ambari to 2.5.2.0-134
> * Registered HDP 2.6.1.0 repo, installed packages
> * Restarted services that needed restart
> * Ran service checks
> * Started upgrade
> 
> Result: _Restarting History Server_ step failed with 
> 
> {noformat:title=errors-91.txt}
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 134, in <module>
>     HistoryServer().execute()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
>     method(env)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 841, in restart
>     self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
>   File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 85, in pre_upgrade_restart
>     copy_to_hdfs("mapreduce", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 256, in copy_to_hdfs
>     mode=0555
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 560, in action_create_on_execute
>     self.action_delayed("create")
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 550, in action_delayed
>     parsed_not_managed_paths = [HdfsResourceProvider.parse_path(path) for path in self.resource.immutable_paths]
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 519, in parse_path
>     math_with_protocol_and_nn_url = re.match("[a-zA-Z]+://[^/]+(/.+)", path)
>   File "/usr/lib64/python2.7/re.py", line 137, in match
>     return _compile(pattern, flags).match(string)
> TypeError: expected string or buffer
> {noformat}
> 
> {noformat:title=command-91.json}
> "hostLevelParams": {
> ...
> "not_managed_hdfs_path_list": "[null,\"/mr-history/done\",\"/app-logs\"]",
> ...
> }
> {noformat}
> 
> Failure seems to be due to {{null}} value in the list.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 9b19ade 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 82171f6 
>   ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java daca2d0 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java ab8026c 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py 9a9f9ca 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py 6495209 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java b1c10f5 
> 
> 
> Diff: https://reviews.apache.org/r/60921/diff/1/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>