You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2014/11/08 19:15:10 UTC
Review Request 27776: Couldn't find any package by regex
'hadooplzo-2-1-.*'
when HDFS Client installing at HDP 2.1 or 2.0 Stacks
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27776/
-----------------------------------------------------------
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-8229
https://issues.apache.org/jira/browse/AMBARI-8229
Repository: ambari
Description
-------
Example:
<http://ec2-54-173-10-218.compute-1.amazonaws.com:8080/#/installer/step9>
STR:
Try to deploy cluster with mentioned parameters with all components at all
hosts.
Actual result:
Have problems with HDFS Client install
stderr:
2014-11-08 15:12:41,063 - Error while executing command 'install':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 123, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 31, in install
self.config(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 48, in config
hdfs()
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py", line 83, in hdfs
Package(params.lzo_packages_for_current_host)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 40, in action_install
self.install_package(package_name)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 35, in wrapper
return function_to_decorate(self, name)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 57, in install_package
shell.checked_call(cmd)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 36, in checked_call
return _call(command, logoutput, True, cwd, env, preexec_fn, user, wait_for_finish, timeout, path)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in _call
raise Fail(err_msg)
Fail: Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package hadooplzo-2-1-.*
E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
stdout:
2014-11-08 15:12:19,944 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] {'environment': ..., 'not_if': 'test -e /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip', 'ignore_failures': True, 'path': ['/bin', '/usr/bin/']}
2014-11-08 15:12:19,966 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] due to not_if
2014-11-08 15:12:19,967 - Group['hadoop'] {'ignore_failures': False}
2014-11-08 15:12:19,970 - Modifying group hadoop
2014-11-08 15:12:20,006 - Group['nobody'] {'ignore_failures': False}
2014-11-08 15:12:20,007 - Modifying group nobody
2014-11-08 15:12:20,039 - Group['users'] {'ignore_failures': False}
2014-11-08 15:12:20,040 - Modifying group users
2014-11-08 15:12:20,074 - Group['nagios'] {'ignore_failures': False}
2014-11-08 15:12:20,075 - Modifying group nagios
2014-11-08 15:12:20,108 - User['hive'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,108 - Modifying user hive
2014-11-08 15:12:20,137 - User['oozie'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
2014-11-08 15:12:20,138 - Modifying user oozie
2014-11-08 15:12:20,164 - User['nobody'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'nobody']}
2014-11-08 15:12:20,165 - Modifying user nobody
2014-11-08 15:12:20,191 - User['nagios'] {'gid': 'nagios', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,192 - Modifying user nagios
2014-11-08 15:12:20,220 - User['ambari-qa'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
2014-11-08 15:12:20,221 - Modifying user ambari-qa
2014-11-08 15:12:20,248 - User['flume'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,249 - Modifying user flume
2014-11-08 15:12:20,276 - User['hdfs'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,277 - Modifying user hdfs
2014-11-08 15:12:20,306 - User['storm'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,307 - Modifying user storm
2014-11-08 15:12:20,338 - User['mapred'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,339 - Modifying user mapred
2014-11-08 15:12:20,367 - User['hbase'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,368 - Modifying user hbase
2014-11-08 15:12:20,395 - User['tez'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
2014-11-08 15:12:20,396 - Modifying user tez
2014-11-08 15:12:20,424 - User['zookeeper'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,425 - Modifying user zookeeper
2014-11-08 15:12:20,453 - User['falcon'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,454 - Modifying user falcon
2014-11-08 15:12:20,480 - User['sqoop'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,481 - Modifying user sqoop
2014-11-08 15:12:20,508 - User['yarn'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,509 - Modifying user yarn
2014-11-08 15:12:20,537 - User['hcat'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2014-11-08 15:12:20,538 - Modifying user hcat
2014-11-08 15:12:20,566 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2014-11-08 15:12:20,569 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] {'not_if': 'test $(id -u ambari-qa) -gt 1000'}
2014-11-08 15:12:20,596 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] due to not_if
2014-11-08 15:12:20,597 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2014-11-08 15:12:20,599 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/mnt/hadoop/hbase 2>/dev/null'] {'not_if': 'test $(id -u hbase) -gt 1000'}
2014-11-08 15:12:20,625 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/mnt/hadoop/hbase 2>/dev/null'] due to not_if
2014-11-08 15:12:20,626 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'root', 'recursive': True}
2014-11-08 15:12:20,627 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
2014-11-08 15:12:20,652 - Skipping Link['/etc/hadoop/conf'] due to not_if
2014-11-08 15:12:20,670 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs'}
2014-11-08 15:12:20,690 - Repository['HDP-2.1'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu12/2.x/BUILDS/2.1.8.0-788', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': 'repo_ubuntu.j2', 'repo_file_name': 'HDP', 'mirror_list': None}
2014-11-08 15:12:20,701 - File['/tmp/tmpUQBCcQ'] {'content': Template('repo_ubuntu.j2')}
2014-11-08 15:12:20,702 - Writing File['/tmp/tmpUQBCcQ'] because contents don't match
2014-11-08 15:12:20,703 - Repository['HDP-UTILS-1.1.0.19'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/ubuntu12', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': 'repo_ubuntu.j2', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
2014-11-08 15:12:20,706 - File['/tmp/tmpl4K4Xd'] {'content': Template('repo_ubuntu.j2')}
2014-11-08 15:12:20,706 - Writing File['/tmp/tmpl4K4Xd'] because contents don't match
2014-11-08 15:12:20,707 - Package['unzip'] {}
2014-11-08 15:12:20,768 - Installing package unzip ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install unzip')
2014-11-08 15:12:21,453 - Package['curl'] {}
2014-11-08 15:12:21,500 - Installing package curl ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install curl')
2014-11-08 15:12:22,169 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/ ; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//jdk-7u67-linux-x64.tar.gz -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz'] {'environment': ..., 'not_if': 'test -e /usr/jdk64/jdk1.7.0_67/bin/java', 'path': ['/bin', '/usr/bin/']}
2014-11-08 15:12:22,189 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/ ; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//jdk-7u67-linux-x64.tar.gz -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz'] due to not_if
2014-11-08 15:12:22,191 - Execute['mkdir -p /usr/jdk64 ; cd /usr/jdk64 ; tar -xf /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz > /dev/null 2>&1'] {'not_if': 'test -e /usr/jdk64/jdk1.7.0_67/bin/java', 'path': ['/bin', '/usr/bin/']}
2014-11-08 15:12:22,210 - Skipping Execute['mkdir -p /usr/jdk64 ; cd /usr/jdk64 ; tar -xf /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz > /dev/null 2>&1'] due to not_if
2014-11-08 15:12:22,438 - Package['hadoop'] {}
2014-11-08 15:12:22,486 - Installing package hadoop ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop')
2014-11-08 15:12:23,184 - Package['hadoop-lzo'] {}
2014-11-08 15:12:23,247 - Installing package hadoop-lzo ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-lzo')
2014-11-08 15:12:24,000 - Package['libsnappy1'] {}
2014-11-08 15:12:24,079 - Installing package libsnappy1 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libsnappy1')
2014-11-08 15:12:24,867 - Package['libsnappy-dev'] {}
2014-11-08 15:12:24,914 - Installing package libsnappy-dev ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libsnappy-dev')
2014-11-08 15:12:25,598 - Package['liblzo2-2'] {}
2014-11-08 15:12:25,646 - Installing package liblzo2-2 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install liblzo2-2')
2014-11-08 15:12:26,373 - Package['hadoop-hdfs'] {}
2014-11-08 15:12:26,435 - Installing package hadoop-hdfs ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-hdfs')
2014-11-08 15:12:27,166 - Package['libhdfs0'] {}
2014-11-08 15:12:27,212 - Installing package libhdfs0 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libhdfs0')
2014-11-08 15:12:27,943 - Package['libhdfs0-dev'] {}
2014-11-08 15:12:27,990 - Installing package libhdfs0-dev ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libhdfs0-dev')
2014-11-08 15:12:28,696 - Directory['/etc/security/limits.d'] {'owner': 'root', 'group': 'root', 'recursive': True}
2014-11-08 15:12:28,707 - File['/etc/security/limits.d/hdfs.conf'] {'content': Template('hdfs.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2014-11-08 15:12:28,708 - XmlConfig['hadoop-policy.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
2014-11-08 15:12:28,724 - Generating config: /etc/hadoop/conf/hadoop-policy.xml
2014-11-08 15:12:28,724 - File['/etc/hadoop/conf/hadoop-policy.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2014-11-08 15:12:28,726 - Writing File['/etc/hadoop/conf/hadoop-policy.xml'] because contents don't match
2014-11-08 15:12:28,726 - XmlConfig['hdfs-site.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
2014-11-08 15:12:28,739 - Generating config: /etc/hadoop/conf/hdfs-site.xml
2014-11-08 15:12:28,740 - File['/etc/hadoop/conf/hdfs-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2014-11-08 15:12:28,742 - Writing File['/etc/hadoop/conf/hdfs-site.xml'] because contents don't match
2014-11-08 15:12:28,742 - XmlConfig['core-site.xml'] {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': ..., 'owner': 'hdfs', 'configurations': ...}
2014-11-08 15:12:28,755 - Generating config: /etc/hadoop/conf/core-site.xml
2014-11-08 15:12:28,755 - File['/etc/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2014-11-08 15:12:28,757 - Writing File['/etc/hadoop/conf/core-site.xml'] because contents don't match
2014-11-08 15:12:28,759 - File['/etc/hadoop/conf/slaves'] {'content': Template('slaves.j2'), 'owner': 'hdfs'}
2014-11-08 15:12:28,760 - Package['hadoop-lzo'] {}
2014-11-08 15:12:28,823 - Installing package hadoop-lzo ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-lzo')
2014-11-08 15:12:29,514 - Package['liblzo2-2'] {}
2014-11-08 15:12:29,562 - Installing package liblzo2-2 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install liblzo2-2')
2014-11-08 15:12:30,278 - Package['hadooplzo-2-1-.*'] {}
2014-11-08 15:12:30,327 - Installing package hadooplzo-2-1-.* ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*')
2014-11-08 15:12:30,941 - Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package hadooplzo-2-1-.*
E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
2014-11-08 15:12:30,942 - Failed to install package hadooplzo-2-1-.*. Executing `apt-get update -qq`
2014-11-08 15:12:40,478 - Retrying to install package hadooplzo-2-1-.*
2014-11-08 15:12:41,063 - Error while executing command 'install':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 123, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 31, in install
self.config(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 48, in config
hdfs()
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py", line 83, in hdfs
Package(params.lzo_packages_for_current_host)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 40, in action_install
self.install_package(package_name)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 35, in wrapper
return function_to_decorate(self, name)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 57, in install_package
shell.checked_call(cmd)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 36, in checked_call
return _call(command, logoutput, True, cwd, env, preexec_fn, user, wait_for_finish, timeout, path)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in _call
raise Fail(err_msg)
Fail: Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package hadooplzo-2-1-.*
E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
Expected result:
Deploy end successfully.
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/params.py 5cd15ae
Diff: https://reviews.apache.org/r/27776/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 27776: Couldn't find any package by regex
'hadooplzo-2-1-.*' when HDFS Client installing at HDP 2.1 or 2.0 Stacks
Posted by Myroslav Papirkovskyy <mp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27776/#review60488
-----------------------------------------------------------
Ship it!
Ship It!
- Myroslav Papirkovskyy
On Лис. 8, 2014, 8:15 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27776/
> -----------------------------------------------------------
>
> (Updated Лис. 8, 2014, 8:15 після полудня)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-8229
> https://issues.apache.org/jira/browse/AMBARI-8229
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Example:
> <http://ec2-54-173-10-218.compute-1.amazonaws.com:8080/#/installer/step9>
>
> STR:
> Try to deploy cluster with mentioned parameters with all components at all
> hosts.
>
> Actual result:
> Have problems with HDFS Client install
>
>
>
>
> stderr:
> 2014-11-08 15:12:41,063 - Error while executing command 'install':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 123, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 31, in install
> self.config(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 48, in config
> hdfs()
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py", line 83, in hdfs
> Package(params.lzo_packages_for_current_host)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 40, in action_install
> self.install_package(package_name)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 35, in wrapper
> return function_to_decorate(self, name)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 57, in install_package
> shell.checked_call(cmd)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 36, in checked_call
> return _call(command, logoutput, True, cwd, env, preexec_fn, user, wait_for_finish, timeout, path)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in _call
> raise Fail(err_msg)
> Fail: Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
> Reading package lists...
> Building dependency tree...
> Reading state information...
> E: Unable to locate package hadooplzo-2-1-.*
> E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
> stdout:
> 2014-11-08 15:12:19,944 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] {'environment': ..., 'not_if': 'test -e /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip', 'ignore_failures': True, 'path': ['/bin', '/usr/bin/']}
> 2014-11-08 15:12:19,966 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//UnlimitedJCEPolicyJDK7.zip -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip'] due to not_if
> 2014-11-08 15:12:19,967 - Group['hadoop'] {'ignore_failures': False}
> 2014-11-08 15:12:19,970 - Modifying group hadoop
> 2014-11-08 15:12:20,006 - Group['nobody'] {'ignore_failures': False}
> 2014-11-08 15:12:20,007 - Modifying group nobody
> 2014-11-08 15:12:20,039 - Group['users'] {'ignore_failures': False}
> 2014-11-08 15:12:20,040 - Modifying group users
> 2014-11-08 15:12:20,074 - Group['nagios'] {'ignore_failures': False}
> 2014-11-08 15:12:20,075 - Modifying group nagios
> 2014-11-08 15:12:20,108 - User['hive'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,108 - Modifying user hive
> 2014-11-08 15:12:20,137 - User['oozie'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
> 2014-11-08 15:12:20,138 - Modifying user oozie
> 2014-11-08 15:12:20,164 - User['nobody'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'nobody']}
> 2014-11-08 15:12:20,165 - Modifying user nobody
> 2014-11-08 15:12:20,191 - User['nagios'] {'gid': 'nagios', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,192 - Modifying user nagios
> 2014-11-08 15:12:20,220 - User['ambari-qa'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
> 2014-11-08 15:12:20,221 - Modifying user ambari-qa
> 2014-11-08 15:12:20,248 - User['flume'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,249 - Modifying user flume
> 2014-11-08 15:12:20,276 - User['hdfs'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,277 - Modifying user hdfs
> 2014-11-08 15:12:20,306 - User['storm'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,307 - Modifying user storm
> 2014-11-08 15:12:20,338 - User['mapred'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,339 - Modifying user mapred
> 2014-11-08 15:12:20,367 - User['hbase'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,368 - Modifying user hbase
> 2014-11-08 15:12:20,395 - User['tez'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
> 2014-11-08 15:12:20,396 - Modifying user tez
> 2014-11-08 15:12:20,424 - User['zookeeper'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,425 - Modifying user zookeeper
> 2014-11-08 15:12:20,453 - User['falcon'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,454 - Modifying user falcon
> 2014-11-08 15:12:20,480 - User['sqoop'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,481 - Modifying user sqoop
> 2014-11-08 15:12:20,508 - User['yarn'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,509 - Modifying user yarn
> 2014-11-08 15:12:20,537 - User['hcat'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
> 2014-11-08 15:12:20,538 - Modifying user hcat
> 2014-11-08 15:12:20,566 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> 2014-11-08 15:12:20,569 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] {'not_if': 'test $(id -u ambari-qa) -gt 1000'}
> 2014-11-08 15:12:20,596 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 2>/dev/null'] due to not_if
> 2014-11-08 15:12:20,597 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
> 2014-11-08 15:12:20,599 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/mnt/hadoop/hbase 2>/dev/null'] {'not_if': 'test $(id -u hbase) -gt 1000'}
> 2014-11-08 15:12:20,625 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/mnt/hadoop/hbase 2>/dev/null'] due to not_if
> 2014-11-08 15:12:20,626 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'root', 'recursive': True}
> 2014-11-08 15:12:20,627 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
> 2014-11-08 15:12:20,652 - Skipping Link['/etc/hadoop/conf'] due to not_if
> 2014-11-08 15:12:20,670 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs'}
> 2014-11-08 15:12:20,690 - Repository['HDP-2.1'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu12/2.x/BUILDS/2.1.8.0-788', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': 'repo_ubuntu.j2', 'repo_file_name': 'HDP', 'mirror_list': None}
> 2014-11-08 15:12:20,701 - File['/tmp/tmpUQBCcQ'] {'content': Template('repo_ubuntu.j2')}
> 2014-11-08 15:12:20,702 - Writing File['/tmp/tmpUQBCcQ'] because contents don't match
> 2014-11-08 15:12:20,703 - Repository['HDP-UTILS-1.1.0.19'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/ubuntu12', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': 'repo_ubuntu.j2', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
> 2014-11-08 15:12:20,706 - File['/tmp/tmpl4K4Xd'] {'content': Template('repo_ubuntu.j2')}
> 2014-11-08 15:12:20,706 - Writing File['/tmp/tmpl4K4Xd'] because contents don't match
> 2014-11-08 15:12:20,707 - Package['unzip'] {}
> 2014-11-08 15:12:20,768 - Installing package unzip ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install unzip')
> 2014-11-08 15:12:21,453 - Package['curl'] {}
> 2014-11-08 15:12:21,500 - Installing package curl ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install curl')
> 2014-11-08 15:12:22,169 - Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/ ; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//jdk-7u67-linux-x64.tar.gz -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz'] {'environment': ..., 'not_if': 'test -e /usr/jdk64/jdk1.7.0_67/bin/java', 'path': ['/bin', '/usr/bin/']}
> 2014-11-08 15:12:22,189 - Skipping Execute['mkdir -p /var/lib/ambari-agent/data/tmp/AMBARI-artifacts/ ; curl -kf -x "" --retry 10 http://ip-172-31-46-254.ec2.internal:8080/resources//jdk-7u67-linux-x64.tar.gz -o /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz'] due to not_if
> 2014-11-08 15:12:22,191 - Execute['mkdir -p /usr/jdk64 ; cd /usr/jdk64 ; tar -xf /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz > /dev/null 2>&1'] {'not_if': 'test -e /usr/jdk64/jdk1.7.0_67/bin/java', 'path': ['/bin', '/usr/bin/']}
> 2014-11-08 15:12:22,210 - Skipping Execute['mkdir -p /usr/jdk64 ; cd /usr/jdk64 ; tar -xf /var/lib/ambari-agent/data/tmp/AMBARI-artifacts//jdk-7u67-linux-x64.tar.gz > /dev/null 2>&1'] due to not_if
> 2014-11-08 15:12:22,438 - Package['hadoop'] {}
> 2014-11-08 15:12:22,486 - Installing package hadoop ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop')
> 2014-11-08 15:12:23,184 - Package['hadoop-lzo'] {}
> 2014-11-08 15:12:23,247 - Installing package hadoop-lzo ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-lzo')
> 2014-11-08 15:12:24,000 - Package['libsnappy1'] {}
> 2014-11-08 15:12:24,079 - Installing package libsnappy1 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libsnappy1')
> 2014-11-08 15:12:24,867 - Package['libsnappy-dev'] {}
> 2014-11-08 15:12:24,914 - Installing package libsnappy-dev ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libsnappy-dev')
> 2014-11-08 15:12:25,598 - Package['liblzo2-2'] {}
> 2014-11-08 15:12:25,646 - Installing package liblzo2-2 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install liblzo2-2')
> 2014-11-08 15:12:26,373 - Package['hadoop-hdfs'] {}
> 2014-11-08 15:12:26,435 - Installing package hadoop-hdfs ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-hdfs')
> 2014-11-08 15:12:27,166 - Package['libhdfs0'] {}
> 2014-11-08 15:12:27,212 - Installing package libhdfs0 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libhdfs0')
> 2014-11-08 15:12:27,943 - Package['libhdfs0-dev'] {}
> 2014-11-08 15:12:27,990 - Installing package libhdfs0-dev ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install libhdfs0-dev')
> 2014-11-08 15:12:28,696 - Directory['/etc/security/limits.d'] {'owner': 'root', 'group': 'root', 'recursive': True}
> 2014-11-08 15:12:28,707 - File['/etc/security/limits.d/hdfs.conf'] {'content': Template('hdfs.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
> 2014-11-08 15:12:28,708 - XmlConfig['hadoop-policy.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
> 2014-11-08 15:12:28,724 - Generating config: /etc/hadoop/conf/hadoop-policy.xml
> 2014-11-08 15:12:28,724 - File['/etc/hadoop/conf/hadoop-policy.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
> 2014-11-08 15:12:28,726 - Writing File['/etc/hadoop/conf/hadoop-policy.xml'] because contents don't match
> 2014-11-08 15:12:28,726 - XmlConfig['hdfs-site.xml'] {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': ..., 'configurations': ...}
> 2014-11-08 15:12:28,739 - Generating config: /etc/hadoop/conf/hdfs-site.xml
> 2014-11-08 15:12:28,740 - File['/etc/hadoop/conf/hdfs-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
> 2014-11-08 15:12:28,742 - Writing File['/etc/hadoop/conf/hdfs-site.xml'] because contents don't match
> 2014-11-08 15:12:28,742 - XmlConfig['core-site.xml'] {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': ..., 'owner': 'hdfs', 'configurations': ...}
> 2014-11-08 15:12:28,755 - Generating config: /etc/hadoop/conf/core-site.xml
> 2014-11-08 15:12:28,755 - File['/etc/hadoop/conf/core-site.xml'] {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
> 2014-11-08 15:12:28,757 - Writing File['/etc/hadoop/conf/core-site.xml'] because contents don't match
> 2014-11-08 15:12:28,759 - File['/etc/hadoop/conf/slaves'] {'content': Template('slaves.j2'), 'owner': 'hdfs'}
> 2014-11-08 15:12:28,760 - Package['hadoop-lzo'] {}
> 2014-11-08 15:12:28,823 - Installing package hadoop-lzo ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadoop-lzo')
> 2014-11-08 15:12:29,514 - Package['liblzo2-2'] {}
> 2014-11-08 15:12:29,562 - Installing package liblzo2-2 ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install liblzo2-2')
> 2014-11-08 15:12:30,278 - Package['hadooplzo-2-1-.*'] {}
> 2014-11-08 15:12:30,327 - Installing package hadooplzo-2-1-.* ('DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*')
> 2014-11-08 15:12:30,941 - Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
> Reading package lists...
> Building dependency tree...
> Reading state information...
> E: Unable to locate package hadooplzo-2-1-.*
> E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
> 2014-11-08 15:12:30,942 - Failed to install package hadooplzo-2-1-.*. Executing `apt-get update -qq`
> 2014-11-08 15:12:40,478 - Retrying to install package hadooplzo-2-1-.*
> 2014-11-08 15:12:41,063 - Error while executing command 'install':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 123, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 31, in install
> self.config(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py", line 48, in config
> hdfs()
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py", line 83, in hdfs
> Package(params.lzo_packages_for_current_host)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 40, in action_install
> self.install_package(package_name)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 35, in wrapper
> return function_to_decorate(self, name)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/apt.py", line 57, in install_package
> shell.checked_call(cmd)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 36, in checked_call
> return _call(command, logoutput, True, cwd, env, preexec_fn, user, wait_for_finish, timeout, path)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in _call
> raise Fail(err_msg)
> Fail: Execution of 'DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get -q -o Dpkg::Options::='--force-confdef' --allow-unauthenticated --assume-yes install hadooplzo-2-1-.*' returned 100. stdin: is not a tty
> Reading package lists...
> Building dependency tree...
> Reading state information...
> E: Unable to locate package hadooplzo-2-1-.*
> E: Couldn't find any package by regex 'hadooplzo-2-1-.*'
>
>
> Expected result:
> Deploy end successfully.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/params.py 5cd15ae
>
> Diff: https://reviews.apache.org/r/27776/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>