You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2017/01/29 12:27:11 UTC
Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs
-----
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
test on live cluster. Still there are things to check
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Nate Cole <nc...@hortonworks.com>.
> On Jan. 30, 2017, 1:50 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, lines 141-142
> > <https://reviews.apache.org/r/56059/diff/1/?file=1618458#file1618458line141>
> >
> > What about the installers for other OSes?
> > Can we have a dictionary/config with the installer for the current OS?
>
> Dmitro Lisnichenko wrote:
> As of now, we are only aware of issues with yum/rpm. On the other hand, we already agreed with Nate to have additional argument with dictionary of process name-graceful delay for it (for now, it will be only populated with yum and rpm entries)
I think that could be a future consideration. The bug is for yum/rpm specifically. Other OS'es would have different binaries, so keying off that may not get us anything.
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163543
-----------------------------------------------------------
On Feb. 1, 2017, 2:13 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 1, 2017, 2:13 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Jan. 30, 2017, 8:50 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, lines 141-142
> > <https://reviews.apache.org/r/56059/diff/1/?file=1618458#file1618458line141>
> >
> > What about the installers for other OSes?
> > Can we have a dictionary/config with the installer for the current OS?
As of now, we are only aware of issues with yum/rpm. On the other hand, we already agreed with Nate to have additional argument with dictionary of process name-graceful delay for it (for now, it will be only populated with yum and rpm entries)
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163543
-----------------------------------------------------------
On Jan. 29, 2017, 2:27 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 2:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163543
-----------------------------------------------------------
ambari-common/src/main/python/ambari_commons/shell.py (lines 130 - 131)
<https://reviews.apache.org/r/56059/#comment234991>
What about the installers for other OSes?
Can we have a dictionary/config with the installer for the current OS?
- Alejandro Fernandez
On Jan. 29, 2017, 12:27 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 12:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Nate Cole <nc...@hortonworks.com>.
> On Jan. 29, 2017, 5:09 p.m., Nate Cole wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, lines 141-142
> > <https://reviews.apache.org/r/56059/diff/1/?file=1618458#file1618458line141>
> >
> > Seems awfully specific. Would be more flexible if we could pass in the kill delay as an argument whose default is 5 (the graceful_kill_delay).
>
> Dmitro Lisnichenko wrote:
> Do you mean having a single graceful_kill_delay parameter (the time between SIGTERM and SIGKILL)? Or you mean exposing yum_rpm_gracefull_kill_delay parameter for timeout between sending SIGTERM to sensitive process like yum, and sending SIGKILL?
What I mean is that yumrpm.py is the one building the command to run. I would prefer supplying the timeout there instead of using the yum_rpm_present as it's just something else we have to remember that we did 6 months from now.
We already know when we're calling yum. We have a ton of arguments to control the timeout behavior, can we just be using them? Can we use try_sleep, timeout, on_timeout, timeout_kill_strategy etc to our advantage (see shell.py/checked_call for complete list).
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163443
-----------------------------------------------------------
On Jan. 29, 2017, 7:27 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 7:27 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Jan. 30, 2017, 12:09 a.m., Nate Cole wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, lines 141-142
> > <https://reviews.apache.org/r/56059/diff/1/?file=1618458#file1618458line141>
> >
> > Seems awfully specific. Would be more flexible if we could pass in the kill delay as an argument whose default is 5 (the graceful_kill_delay).
Do you mean having a single graceful_kill_delay parameter (the time between SIGTERM and SIGKILL)? Or you mean exposing yum_rpm_gracefull_kill_delay parameter for timeout between sending SIGTERM to sensitive process like yum, and sending SIGKILL?
> On Jan. 30, 2017, 12:09 a.m., Nate Cole wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, lines 127-128
> > <https://reviews.apache.org/r/56059/diff/1/?file=1618458#file1618458line127>
> >
> > Will this change work when running as non-root?
this code works under the same user as user that launches process. So permissions logic should not change because of this change
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163443
-----------------------------------------------------------
On Jan. 29, 2017, 2:27 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 2:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163443
-----------------------------------------------------------
ambari-common/src/main/python/ambari_commons/shell.py (lines 116 - 117)
<https://reviews.apache.org/r/56059/#comment234906>
Will this change work when running as non-root?
ambari-common/src/main/python/ambari_commons/shell.py (lines 130 - 131)
<https://reviews.apache.org/r/56059/#comment234907>
Seems awfully specific. Would be more flexible if we could pass in the kill delay as an argument whose default is 5 (the graceful_kill_delay).
- Nate Cole
On Jan. 29, 2017, 7:27 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 7:27 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164021
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Feb. 2, 2017, 11:16 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2017, 11:16 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Package Installation fails due to error in
Berkeley DB library
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164372
-----------------------------------------------------------
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py (line 38)
<https://reviews.apache.org/r/56059/#comment236068>
We're no longer supporting Windows so I would actually leave those annotations out.
- Alejandro Fernandez
On Feb. 6, 2017, 12:46 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 6, 2017, 12:46 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review165112
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On Feb. 10, 2017, 10:13 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2017, 10:13 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This single liner does not work correctly in some cases
> {code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
> """ ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
> """K=P } P && !/_/ { P="" } END { print "kill -""" \
> + str(signal) + """ "K }' | sh """
> {code}
>
> *Example*
> {code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
>
> ps xf
>
> 3231 ? Ss 0:01 \_ sshd: root@pts/0
> 3233 pts/0 Ss 0:00 | \_ -bash
> 17984 pts/0 S+ 0:00 | \_ -bash
> 17985 pts/0 S+ 0:00 | \_ sleep 3141592
> 17986 pts/0 S+ 0:00 | \_ -bash
> 17987 pts/0 S+ 0:00 | \_ sleep 3141592
> 17988 pts/0 S+ 0:00 | \_ sleep 3141592
> 17738 ? Ss 0:00 \_ sshd: root@pts/1
> 17740 pts/1 Ss 0:00 \_ -bash
> 17989 pts/1 R+ 0:00 \_ ps xf
>
> ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
>
> (PID=17987)
> result : "kill 17987 17988 17738 17740 18083 18084"
> but right will only "kill 17987"
>
> (PID=17985)
> result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
> right : "kill 17985"
>
> (PID=17986)
> result : "kill 17986 17987 17988 17738 17740 18980 18981"
> right : "kill 17986 17987 17988"
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 10, 2017, 3:15 p.m.)
Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
This single liner does not work correctly in some cases
{code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
""" ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
"""K=P } P && !/_/ { P="" } END { print "kill -""" \
+ str(signal) + """ "K }' | sh """
{code}
*Example*
{code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
ps xf
3231 ? Ss 0:01 \_ sshd: root@pts/0
3233 pts/0 Ss 0:00 | \_ -bash
17984 pts/0 S+ 0:00 | \_ -bash
17985 pts/0 S+ 0:00 | \_ sleep 3141592
17986 pts/0 S+ 0:00 | \_ -bash
17987 pts/0 S+ 0:00 | \_ sleep 3141592
17988 pts/0 S+ 0:00 | \_ sleep 3141592
17738 ? Ss 0:00 \_ sshd: root@pts/1
17740 pts/1 Ss 0:00 \_ -bash
17989 pts/1 R+ 0:00 \_ ps xf
ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
(PID=17987)
result : "kill 17987 17988 17738 17740 18083 18084"
but right will only "kill 17987"
(PID=17985)
result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
right : "kill 17985"
(PID=17986)
result : "kill 17986 17987 17988 17738 17740 18980 18981"
right : "kill 17986 17987 17988"
{code}
Diffs
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 10, 2017, 12:13 p.m.)
Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Fixes
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
This single liner does not work correctly in some cases
{code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
""" ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
"""K=P } P && !/_/ { P="" } END { print "kill -""" \
+ str(signal) + """ "K }' | sh """
{code}
*Example*
{code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
ps xf
3231 ? Ss 0:01 \_ sshd: root@pts/0
3233 pts/0 Ss 0:00 | \_ -bash
17984 pts/0 S+ 0:00 | \_ -bash
17985 pts/0 S+ 0:00 | \_ sleep 3141592
17986 pts/0 S+ 0:00 | \_ -bash
17987 pts/0 S+ 0:00 | \_ sleep 3141592
17988 pts/0 S+ 0:00 | \_ sleep 3141592
17738 ? Ss 0:00 \_ sshd: root@pts/1
17740 pts/1 Ss 0:00 \_ -bash
17989 pts/1 R+ 0:00 \_ ps xf
ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
(PID=17987)
result : "kill 17987 17988 17738 17740 18083 18084"
but right will only "kill 17987"
(PID=17985)
result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
right : "kill 17985"
(PID=17986)
result : "kill 17986 17987 17988 17738 17740 18980 18981"
right : "kill 17986 17987 17988"
{code}
Diffs (updated)
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Feb. 8, 2017, 4:57 p.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/process_utils.py, line 48
> > <https://reviews.apache.org/r/56059/diff/5/?file=1627633#file1627633line48>
> >
> > Was this tested in non-root?
> > Should we maybe call it with ambari-sudo.sh to be able to kill sub-processes started as service users hdfs etc. when running in non-root
>
> Nate Cole wrote:
> +1 on this
yes
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164687
-----------------------------------------------------------
On Feb. 10, 2017, 12:13 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2017, 12:13 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This single liner does not work correctly in some cases
> {code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
> """ ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
> """K=P } P && !/_/ { P="" } END { print "kill -""" \
> + str(signal) + """ "K }' | sh """
> {code}
>
> *Example*
> {code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
>
> ps xf
>
> 3231 ? Ss 0:01 \_ sshd: root@pts/0
> 3233 pts/0 Ss 0:00 | \_ -bash
> 17984 pts/0 S+ 0:00 | \_ -bash
> 17985 pts/0 S+ 0:00 | \_ sleep 3141592
> 17986 pts/0 S+ 0:00 | \_ -bash
> 17987 pts/0 S+ 0:00 | \_ sleep 3141592
> 17988 pts/0 S+ 0:00 | \_ sleep 3141592
> 17738 ? Ss 0:00 \_ sshd: root@pts/1
> 17740 pts/1 Ss 0:00 \_ -bash
> 17989 pts/1 R+ 0:00 \_ ps xf
>
> ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
>
> (PID=17987)
> result : "kill 17987 17988 17738 17740 18083 18084"
> but right will only "kill 17987"
>
> (PID=17985)
> result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
> right : "kill 17985"
>
> (PID=17986)
> result : "kill 17986 17987 17988 17738 17740 18980 18981"
> right : "kill 17986 17987 17988"
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Nate Cole <nc...@hortonworks.com>.
> On Feb. 8, 2017, 9:57 a.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/process_utils.py, line 48
> > <https://reviews.apache.org/r/56059/diff/5/?file=1627633#file1627633line48>
> >
> > Was this tested in non-root?
> > Should we maybe call it with ambari-sudo.sh to be able to kill sub-processes started as service users hdfs etc. when running in non-root
+1 on this
> On Feb. 8, 2017, 9:57 a.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/ambari_commons/process_utils.py, line 30
> > <https://reviews.apache.org/r/56059/diff/5/?file=1627633#file1627633line30>
> >
> > Can we check the return code of every command which we run here. And give an informative exception in case it is non-zero.
> > Saying which command failed with which output and which return code.
+1 for this as long as we don't fill the logs (we shouldn't be, but you never know)
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164687
-----------------------------------------------------------
On Feb. 8, 2017, 9:50 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 8, 2017, 9:50 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This single liner does not work correctly in some cases
> {code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
> """ ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
> """K=P } P && !/_/ { P="" } END { print "kill -""" \
> + str(signal) + """ "K }' | sh """
> {code}
>
> *Example*
> {code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
>
> ps xf
>
> 3231 ? Ss 0:01 \_ sshd: root@pts/0
> 3233 pts/0 Ss 0:00 | \_ -bash
> 17984 pts/0 S+ 0:00 | \_ -bash
> 17985 pts/0 S+ 0:00 | \_ sleep 3141592
> 17986 pts/0 S+ 0:00 | \_ -bash
> 17987 pts/0 S+ 0:00 | \_ sleep 3141592
> 17988 pts/0 S+ 0:00 | \_ sleep 3141592
> 17738 ? Ss 0:00 \_ sshd: root@pts/1
> 17740 pts/1 Ss 0:00 \_ -bash
> 17989 pts/1 R+ 0:00 \_ ps xf
>
> ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
>
> (PID=17987)
> result : "kill 17987 17988 17738 17740 18083 18084"
> but right will only "kill 17987"
>
> (PID=17985)
> result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
> right : "kill 17985"
>
> (PID=17986)
> result : "kill 17986 17987 17988 17738 17740 18980 18981"
> right : "kill 17986 17987 17988"
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164687
-----------------------------------------------------------
ambari-common/src/main/python/ambari_commons/process_utils.py (line 30)
<https://reviews.apache.org/r/56059/#comment236456>
Can we check the return code of every command which we run here. And give an informative exception in case it is non-zero.
Saying which command failed with which output and which return code.
ambari-common/src/main/python/ambari_commons/process_utils.py (line 48)
<https://reviews.apache.org/r/56059/#comment236455>
Was this tested in non-root?
Should we maybe call it with ambari-sudo.sh to be able to kill sub-processes started as service users hdfs etc. when running in non-root
- Andrew Onischuk
On Feb. 8, 2017, 2:50 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 8, 2017, 2:50 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This single liner does not work correctly in some cases
> {code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
> """ ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
> """K=P } P && !/_/ { P="" } END { print "kill -""" \
> + str(signal) + """ "K }' | sh """
> {code}
>
> *Example*
> {code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
>
> ps xf
>
> 3231 ? Ss 0:01 \_ sshd: root@pts/0
> 3233 pts/0 Ss 0:00 | \_ -bash
> 17984 pts/0 S+ 0:00 | \_ -bash
> 17985 pts/0 S+ 0:00 | \_ sleep 3141592
> 17986 pts/0 S+ 0:00 | \_ -bash
> 17987 pts/0 S+ 0:00 | \_ sleep 3141592
> 17988 pts/0 S+ 0:00 | \_ sleep 3141592
> 17738 ? Ss 0:00 \_ sshd: root@pts/1
> 17740 pts/1 Ss 0:00 \_ -bash
> 17989 pts/1 R+ 0:00 \_ ps xf
>
> ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
>
> (PID=17987)
> result : "kill 17987 17988 17738 17740 18083 18084"
> but right will only "kill 17987"
>
> (PID=17985)
> result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
> right : "kill 17985"
>
> (PID=17986)
> result : "kill 17986 17987 17988 17738 17740 18980 18981"
> right : "kill 17986 17987 17988"
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Broken kill_process_with_children shell
single liner
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 8, 2017, 4:50 p.m.)
Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Removed yum&rpm staff, left only python implementation
Summary (updated)
-----------------
Broken kill_process_with_children shell single liner
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description (updated)
-------
This single liner does not work correctly in some cases
{code} CMD = """ps xf | awk -v PID=""" + str(pid) + \
""" ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;""" + \
"""K=P } P && !/_/ { P="" } END { print "kill -""" \
+ str(signal) + """ "K }' | sh """
{code}
*Example*
{code}((sleep 3141592) & ((sleep 3141592) & (sleep 3141592))
ps xf
3231 ? Ss 0:01 \_ sshd: root@pts/0
3233 pts/0 Ss 0:00 | \_ -bash
17984 pts/0 S+ 0:00 | \_ -bash
17985 pts/0 S+ 0:00 | \_ sleep 3141592
17986 pts/0 S+ 0:00 | \_ -bash
17987 pts/0 S+ 0:00 | \_ sleep 3141592
17988 pts/0 S+ 0:00 | \_ sleep 3141592
17738 ? Ss 0:00 \_ sshd: root@pts/1
17740 pts/1 Ss 0:00 \_ -bash
17989 pts/1 R+ 0:00 \_ ps xf
ps xf | awk -v PID=17987 ' $1 == PID { P = $1; next } P && /_/ { P = P " " $1;K=P } P && !/_/ { P="" } END { print "kill "K }'
(PID=17987)
result : "kill 17987 17988 17738 17740 18083 18084"
but right will only "kill 17987"
(PID=17985)
result : "kill 17985 17986 17987 17988 17738 17740 18697 18698"
right : "kill 17985"
(PID=17986)
result : "kill 17986 17987 17988 17738 17740 18980 18981"
right : "kill 17986 17987 17988"
{code}
Diffs (updated)
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Package Installation fails due to error in
Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 2:46 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing (updated)
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Package Installation fails due to error in
Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 2:41 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Summary (updated)
-----------------
Package Installation fails due to error in Berkeley DB library
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
will add unit tests
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 2:40 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Added unit tests and fixes some existing ones
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs (updated)
-----
ambari-agent/src/test/python/ambari_agent/TestProcessUtils.py PRE-CREATION
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
will add unit tests
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164024
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Feb. 2, 2017, 11:16 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2017, 11:16 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review164007
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Feb. 2, 2017, 4:16 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2017, 4:16 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 2, 2017, 6:16 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Fixed existing tests, still adding new tests
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs (updated)
-----
ambari-agent/src/test/python/ambari_agent/TestShell.py 8d375e3915
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599df8
Diff: https://reviews.apache.org/r/56059/diff/
Testing
-------
will add unit tests
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Feb. 1, 2017, 11:40 p.m., Nate Cole wrote:
> > ambari-common/src/main/python/ambari_commons/shell.py, line 42
> > <https://reviews.apache.org/r/56059/diff/2/?file=1621590#file1621590line42>
> >
> > Is it possible that these paths are different between redhat6 and redhat7 (or any of their flavors?)
I think no, /bin and /usr/bin folders are usually dedicated to categories of binaries
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163886
-----------------------------------------------------------
On Feb. 1, 2017, 9:13 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 1, 2017, 9:13 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163886
-----------------------------------------------------------
ambari-common/src/main/python/ambari_commons/shell.py (line 36)
<https://reviews.apache.org/r/56059/#comment235364>
Is it possible that these paths are different between redhat6 and redhat7 (or any of their flavors?)
- Nate Cole
On Feb. 1, 2017, 2:13 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Feb. 1, 2017, 2:13 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> will add unit tests
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/
-----------------------------------------------------------
(Updated Feb. 1, 2017, 9:13 p.m.)
Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
Changes
-------
changed approach a bit
Bugs: AMBARI-19768
https://issues.apache.org/jira/browse/AMBARI-19768
Repository: ambari
Description
-------
*Steps*
# Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
# Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
# Register HDP-2.6.0.0-216
# Start package installation
*Result:*
Got below errors:
{code}
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
{code}
Diffs (updated)
-----
ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
ambari-common/src/main/python/ambari_commons/shell.py 8d26599
Diff: https://reviews.apache.org/r/56059/diff/
Testing (updated)
-------
will add unit tests
Thanks,
Dmitro Lisnichenko
Re: Review Request 56059: Preview: Package Installation fails due to
error in Berkeley DB library
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56059/#review163429
-----------------------------------------------------------
ambari-common/src/main/python/ambari_commons/process_utils.py (line 51)
<https://reviews.apache.org/r/56059/#comment234889>
will double-check that this condition works
- Dmitro Lisnichenko
On Jan. 29, 2017, 2:27 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56059/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2017, 2:27 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-19768
> https://issues.apache.org/jira/browse/AMBARI-19768
>
>
> Repository: ambari
>
>
> Description
> -------
>
> *Steps*
> # Deploy HDP-2.5.0.0 with Ambari 2.4.1.0
> # Upgrade ambari to 2.5.0.0-481 (I did not register Falcon library, as the jar was already present in /var/lib/ambari-server/resources/je-5.0.73.jar on Ambari server node)
> # Register HDP-2.6.0.0-216
> # Start package installation
>
> *Result:*
> Got below errors:
> {code}
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,419|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 166, in actionexecute
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|ret_code = self.install_packages(package_list)
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 400, in install_packages
> 2016-12-16 13:47:10,420|INFO|MainThread|machine.py:145 - run()|if not verifyDependencies():
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/packages_analyzer.py", line 311, in verifyDependencies
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|code, out = rmf_shell.checked_call(cmd, sudo=True)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|result = function(command, **kwargs)
> 2016-12-16 13:47:10,421|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|result = _call(command, **kwargs_copy)
> 2016-12-16 13:47:10,422|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|raise ExecutionFailed(err_msg, code, out, err)
> 2016-12-16 13:47:10,423|INFO|MainThread|machine.py:145 - run()|ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 check dependencies' returned 1. error: rpmdb: BDB0113 Thread/process 16016/139791567193920 failed: BDB1507 Thread died in Berkeley DB library
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages index using db5 - (-30973)
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|error: cannot open Packages database in /var/lib/rpm
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|CRITICAL:yum.main:
> 2016-12-16 13:47:10,424|INFO|MainThread|machine.py:145 - run()|
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Error: rpmdb open failed
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|Traceback (most recent call last):
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 469, in <module>
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|InstallPackages().execute()
> 2016-12-16 13:47:10,425|INFO|MainThread|machine.py:145 - run()|File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 287, in execute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|method(env)
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|File "/var/lib/ambari-agent/cache/custom_actions/scripts/install_packages.py", line 179, in actionexecute
> 2016-12-16 13:47:10,426|INFO|MainThread|machine.py:145 - run()|raise Fail("Failed to distribute repositories/install packages")
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/ambari_commons/process_utils.py PRE-CREATION
> ambari-common/src/main/python/ambari_commons/shell.py 8d26599
>
> Diff: https://reviews.apache.org/r/56059/diff/
>
>
> Testing
> -------
>
> test on live cluster. Still there are things to check
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>