You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/11/25 18:29:15 UTC
Review Request 40711: Hive Metastore Downgrade Fails Because Of
upgradeSchema Invocation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40711/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Nate Cole, and Swapan Shridhar.
Bugs: AMBARI-14073
https://issues.apache.org/jira/browse/AMBARI-14073
Repository: ambari
Description
-------
When downgrading Hive Metastore on a rolling upgrade, the upgradeSchema function is invoked when it should not be.
{code}
Traceback (most recent call last):
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 239, in <module>
HiveMetastore().execute()
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 218, in execute
method(env)
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 497, in restart
self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 112, in pre_upgrade_restart
self.upgrade_schema(env)
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 235, in upgrade_schema
Execute(command, user=params.hive_user, tries=1, environment=env_dict, logoutput=True)
File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
self.env.run()
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
self.run_action(resource, action)
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
provider_action()
File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 70, in inner
result = function(command, **kwargs)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/hdp/2.2.8.0-3150/hive/bin/schematool -dbType mysql -upgradeSchema' returned 1. org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for upgrade 1.2.0 Metastore schema may be too old or newer
*** schemaTool failed ***
{code}
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py 084494d
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py f2ddc80
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py bdc0fe2
Diff: https://reviews.apache.org/r/40711/diff/
Testing
-------
----------------------------------------------------------------------
Total run:836
Total errors:0
Total failures:0
OK
Thanks,
Jonathan Hurley
Re: Review Request 40711: Hive Metastore Downgrade Fails Because Of
upgradeSchema Invocation
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40711/#review108042
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Nov. 25, 2015, 1:34 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40711/
> -----------------------------------------------------------
>
> (Updated Nov. 25, 2015, 1:34 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-14073
> https://issues.apache.org/jira/browse/AMBARI-14073
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When downgrading Hive Metastore on a rolling upgrade, the upgradeSchema function is invoked when it should not be.
>
> {code}
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 239, in <module>
> HiveMetastore().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 218, in execute
> method(env)
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 497, in restart
> self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 112, in pre_upgrade_restart
> self.upgrade_schema(env)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 235, in upgrade_schema
> Execute(command, user=params.hive_user, tries=1, environment=env_dict, logoutput=True)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 238, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 70, in inner
> result = function(command, **kwargs)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/hdp/2.2.8.0-3150/hive/bin/schematool -dbType mysql -upgradeSchema' returned 1. org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for upgrade 1.2.0 Metastore schema may be too old or newer
> *** schemaTool failed ***
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py 084494d
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 972d532
> ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py bdc0fe2
>
> Diff: https://reviews.apache.org/r/40711/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Total run:836
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 40711: Hive Metastore Downgrade Fails Because Of
upgradeSchema Invocation
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40711/#review108037
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Nov. 25, 2015, 6:34 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40711/
> -----------------------------------------------------------
>
> (Updated Nov. 25, 2015, 6:34 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-14073
> https://issues.apache.org/jira/browse/AMBARI-14073
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When downgrading Hive Metastore on a rolling upgrade, the upgradeSchema function is invoked when it should not be.
>
> {code}
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 239, in <module>
> HiveMetastore().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 218, in execute
> method(env)
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 497, in restart
> self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 112, in pre_upgrade_restart
> self.upgrade_schema(env)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 235, in upgrade_schema
> Execute(command, user=params.hive_user, tries=1, environment=env_dict, logoutput=True)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 238, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 70, in inner
> result = function(command, **kwargs)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/hdp/2.2.8.0-3150/hive/bin/schematool -dbType mysql -upgradeSchema' returned 1. org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for upgrade 1.2.0 Metastore schema may be too old or newer
> *** schemaTool failed ***
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py 084494d
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 972d532
> ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py bdc0fe2
>
> Diff: https://reviews.apache.org/r/40711/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Total run:836
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 40711: Hive Metastore Downgrade Fails Because Of
upgradeSchema Invocation
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40711/
-----------------------------------------------------------
(Updated Nov. 25, 2015, 1:34 p.m.)
Review request for Ambari, Alejandro Fernandez, Nate Cole, and Swapan Shridhar.
Bugs: AMBARI-14073
https://issues.apache.org/jira/browse/AMBARI-14073
Repository: ambari
Description
-------
When downgrading Hive Metastore on a rolling upgrade, the upgradeSchema function is invoked when it should not be.
{code}
Traceback (most recent call last):
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 239, in <module>
HiveMetastore().execute()
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 218, in execute
method(env)
File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 497, in restart
self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 112, in pre_upgrade_restart
self.upgrade_schema(env)
File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 235, in upgrade_schema
Execute(command, user=params.hive_user, tries=1, environment=env_dict, logoutput=True)
File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
self.env.run()
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
self.run_action(resource, action)
File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
provider_action()
File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 70, in inner
result = function(command, **kwargs)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/hdp/2.2.8.0-3150/hive/bin/schematool -dbType mysql -upgradeSchema' returned 1. org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for upgrade 1.2.0 Metastore schema may be too old or newer
*** schemaTool failed ***
{code}
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py 084494d
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 972d532
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py bdc0fe2
Diff: https://reviews.apache.org/r/40711/diff/
Testing
-------
----------------------------------------------------------------------
Total run:836
Total errors:0
Total failures:0
OK
Thanks,
Jonathan Hurley
Re: Review Request 40711: Hive Metastore Downgrade Fails Because Of
upgradeSchema Invocation
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40711/#review108036
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py (lines 105 - 106)
<https://reviews.apache.org/r/40711/#comment167414>
typically we don't use camel case in python.
- Nate Cole
On Nov. 25, 2015, 12:29 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40711/
> -----------------------------------------------------------
>
> (Updated Nov. 25, 2015, 12:29 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Nate Cole, and Swapan Shridhar.
>
>
> Bugs: AMBARI-14073
> https://issues.apache.org/jira/browse/AMBARI-14073
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When downgrading Hive Metastore on a rolling upgrade, the upgradeSchema function is invoked when it should not be.
>
> {code}
> Traceback (most recent call last):
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 239, in <module>
> HiveMetastore().execute()
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 218, in execute
> method(env)
> File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\", line 497, in restart
> self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 112, in pre_upgrade_restart
> self.upgrade_schema(env)
> File \"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py\", line 235, in upgrade_schema
> Execute(command, user=params.hive_user, tries=1, environment=env_dict, logoutput=True)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154, in __init__
> self.env.run()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 158, in run
> self.run_action(resource, action)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\", line 121, in run_action
> provider_action()
> File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\", line 238, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 70, in inner
> result = function(command, **kwargs)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/hdp/2.2.8.0-3150/hive/bin/schematool -dbType mysql -upgradeSchema' returned 1. org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for upgrade 1.2.0 Metastore schema may be too old or newer
> *** schemaTool failed ***
> {code}
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py 084494d
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py f2ddc80
> ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py bdc0fe2
>
> Diff: https://reviews.apache.org/r/40711/diff/
>
>
> Testing
> -------
>
> ----------------------------------------------------------------------
> Total run:836
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>