You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2017/03/11 11:43:31 UTC
Review Request 57536: Restart of Spark2-client fails at setup_spark()
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57536/
-----------------------------------------------------------
Review request for Ambari and Sumit Mohanty.
Bugs: AMBARI-20404
https://issues.apache.org/jira/browse/AMBARI-20404
Repository: ambari
Description
-------
Scenario:
* Added below configs to spark2 default
spark.io.encryption.enabled = true
spark.io.encryption.keySizeBits = 128
spark.io.encryption.keygen.algorithm = HmacSHA1?
* Restart affected services.
Spark2-client is failing to restart with below error.
**stderr**
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 60, in <module>
SparkClient().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 313, in execute
method(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 721, in restart
self.install(env)
File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 36, in install
self.configure(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 116, in locking_configure
original_configure(obj, *args, **kw)
File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 42, in configure
setup_spark(env, 'client', upgrade_type=upgrade_type, action = 'config')
File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py", line 56, in setup_spark
mode=0644
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/properties_file.py", line 54, in action_create
mode = self.resource.mode
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
content = self._get_content()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
return content()
File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
return self.get_content()
File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 143, in get_content
rendered = self.template.render(self.context)
File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
return self.environment.handle_exception(exc_info, True)
File "<template>", line 3, in top-level template code
UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 8: ordinal not in range(128)
ambari-hash : d54803dd86c35f92637a978c58cf06818f8ca0d5
Diffs
-----
ambari-agent/src/test/python/resource_management/TestPropertiesFileResource.py 3dd60e1
ambari-common/src/main/python/resource_management/core/source.py e31f7d6
ambari-common/src/main/python/resource_management/libraries/providers/properties_file.py 823f579
ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py c885d6e
ambari-common/src/main/python/resource_management/libraries/resources/properties_file.py 527b0f6
Diff: https://reviews.apache.org/r/57536/diff/1/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 57536: Restart of Spark2-client fails at
setup_spark()
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On March 11, 2017, 3:27 p.m., Sumit Mohanty wrote:
> > Making this change will allow Ambari to produce the properties file - would it cause trouble when service read it - e.g. when Spark is starting? IOW, do they need to set something in the env file to be able to read UTF-8.
I don't think it would require any change. We have all xml files written in utf-8 for a long and it never was a problem.
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57536/#review168710
-----------------------------------------------------------
On March 11, 2017, 11:43 a.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57536/
> -----------------------------------------------------------
>
> (Updated March 11, 2017, 11:43 a.m.)
>
>
> Review request for Ambari and Sumit Mohanty.
>
>
> Bugs: AMBARI-20404
> https://issues.apache.org/jira/browse/AMBARI-20404
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Scenario:
>
> * Added below configs to spark2 default
>
>
> spark.io.encryption.enabled = true
> spark.io.encryption.keySizeBits = 128
> spark.io.encryption.keygen.algorithm = HmacSHA1?
>
> * Restart affected services.
> Spark2-client is failing to restart with below error.
>
> **stderr**
>
>
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 60, in <module>
> SparkClient().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 313, in execute
> method(env)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 721, in restart
> self.install(env)
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 36, in install
> self.configure(env)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 116, in locking_configure
> original_configure(obj, *args, **kw)
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 42, in configure
> setup_spark(env, 'client', upgrade_type=upgrade_type, action = 'config')
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py", line 56, in setup_spark
> mode=0644
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/properties_file.py", line 54, in action_create
> mode = self.resource.mode
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
> content = self._get_content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
> return content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
> return self.get_content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 143, in get_content
> rendered = self.template.render(self.context)
> File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
> return self.environment.handle_exception(exc_info, True)
> File "<template>", line 3, in top-level template code
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 8: ordinal not in range(128)
>
>
> ambari-hash : d54803dd86c35f92637a978c58cf06818f8ca0d5
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/resource_management/TestPropertiesFileResource.py 3dd60e1
> ambari-common/src/main/python/resource_management/core/source.py e31f7d6
> ambari-common/src/main/python/resource_management/libraries/providers/properties_file.py 823f579
> ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py c885d6e
> ambari-common/src/main/python/resource_management/libraries/resources/properties_file.py 527b0f6
>
>
> Diff: https://reviews.apache.org/r/57536/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 57536: Restart of Spark2-client fails at
setup_spark()
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57536/#review168710
-----------------------------------------------------------
Ship it!
Making this change will allow Ambari to produce the properties file - would it cause trouble when service read it - e.g. when Spark is starting? IOW, do they need to set something in the env file to be able to read UTF-8.
- Sumit Mohanty
On March 11, 2017, 11:43 a.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57536/
> -----------------------------------------------------------
>
> (Updated March 11, 2017, 11:43 a.m.)
>
>
> Review request for Ambari and Sumit Mohanty.
>
>
> Bugs: AMBARI-20404
> https://issues.apache.org/jira/browse/AMBARI-20404
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Scenario:
>
> * Added below configs to spark2 default
>
>
> spark.io.encryption.enabled = true
> spark.io.encryption.keySizeBits = 128
> spark.io.encryption.keygen.algorithm = HmacSHA1?
>
> * Restart affected services.
> Spark2-client is failing to restart with below error.
>
> **stderr**
>
>
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 60, in <module>
> SparkClient().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 313, in execute
> method(env)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 721, in restart
> self.install(env)
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 36, in install
> self.configure(env)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 116, in locking_configure
> original_configure(obj, *args, **kw)
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/spark_client.py", line 42, in configure
> setup_spark(env, 'client', upgrade_type=upgrade_type, action = 'config')
> File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py", line 56, in setup_spark
> mode=0644
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/properties_file.py", line 54, in action_create
> mode = self.resource.mode
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
> content = self._get_content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
> return content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
> return self.get_content()
> File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 143, in get_content
> rendered = self.template.render(self.context)
> File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
> return self.environment.handle_exception(exc_info, True)
> File "<template>", line 3, in top-level template code
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 8: ordinal not in range(128)
>
>
> ambari-hash : d54803dd86c35f92637a978c58cf06818f8ca0d5
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/resource_management/TestPropertiesFileResource.py 3dd60e1
> ambari-common/src/main/python/resource_management/core/source.py e31f7d6
> ambari-common/src/main/python/resource_management/libraries/providers/properties_file.py 823f579
> ambari-common/src/main/python/resource_management/libraries/providers/xml_config.py c885d6e
> ambari-common/src/main/python/resource_management/libraries/resources/properties_file.py 527b0f6
>
>
> Diff: https://reviews.apache.org/r/57536/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>