You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "Abhishek Sinha (JIRA)" <ji...@apache.org> on 2019/04/16 10:56:00 UTC

[jira] [Created] (METRON-2081) Is Metron compatible with newer version of HDP?

Abhishek Sinha created METRON-2081:
--------------------------------------

             Summary: Is Metron compatible with newer version of HDP? 
                 Key: METRON-2081
                 URL: https://issues.apache.org/jira/browse/METRON-2081
             Project: Metron
          Issue Type: Improvement
    Affects Versions: 0.4.1
            Reporter: Abhishek Sinha
             Fix For: 0.4.1


I am unable to deploy HDP 2.5.0 as it gives repo error, can anyone help me with the latest HDP version with which it is compatible with? The error that I receive is as follows: 

 

2019-04-16 15:39:58,855 - Could not find stack selector for stack: HDP

Traceback (most recent call last):

  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py", line 155, in <module>

    ApplicationTimelineServer().execute()

  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 285, in execute

    method(env)

  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py", line 39, in install

    self.install_packages(env)

  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 576, in install_packages

    retry_count=agent_stack_retry_count)

  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__

    self.env.run()

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run

    self.run_action(resource, action)

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action

    provider_action()

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 54, in action_install

    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 51, in install_package

    self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 86, in checked_call_with_retries

    return self._call_with_retries(cmd, is_checked=True, **kwargs)

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 98, in _call_with_retries

    code, out = func(cmd, **kwargs)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner

    result = function(command, **kwargs)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call

    tries=tries, try_sleep=try_sleep)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper

    result = _call(command, **kwargs_copy)

  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call

    raise ExecutionFailed(err_msg, code, out, err)

resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install hadoop_2_5_0_0_1245-yarn' returned 1. Error: Nothing to do

*stdout:*   */var/lib/ambari-agent/data/output-185.txt*

2019-04-16 15:38:35,068 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf

2019-04-16 15:38:35,070 - Group['hadoop'] {}

2019-04-16 15:38:35,071 - Group['users'] {}

2019-04-16 15:38:35,071 - Group['zeppelin'] {}

2019-04-16 15:38:35,071 - Group['metron'] {}

2019-04-16 15:38:35,072 - Group['kibana'] {}

2019-04-16 15:38:35,072 - Group['spark'] {}

2019-04-16 15:38:35,072 - Group['livy'] {}

2019-04-16 15:38:35,072 - Group['elasticsearch'] {}

2019-04-16 15:38:35,072 - User['hive'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,073 - User['zeppelin'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,074 - User['ambari-qa'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}

2019-04-16 15:38:35,074 - User['flume'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,075 - User['metron'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,076 - User['hdfs'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,076 - User['storm'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,077 - User['spark'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,077 - User['kibana'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,078 - User['livy'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,079 - User['mapred'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,079 - User['hbase'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,080 - User['tez'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}

2019-04-16 15:38:35,080 - User['zookeeper'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,081 - User['kafka'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,082 - User['yarn'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,082 - User['hcat'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,083 - User['ams'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,083 - User['elasticsearch'] \{'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}

2019-04-16 15:38:35,084 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] \{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}

2019-04-16 15:38:35,086 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] \{'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}

2019-04-16 15:38:35,105 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if

2019-04-16 15:38:35,106 - Directory['/tmp/hbase-hbase'] \{'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}

2019-04-16 15:38:35,107 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] \{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}

2019-04-16 15:38:35,109 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] \{'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}

2019-04-16 15:38:35,129 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if

2019-04-16 15:38:35,129 - Group['hdfs'] {}

2019-04-16 15:38:35,130 - User['hdfs'] \{'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}

2019-04-16 15:38:35,131 - FS Type:

2019-04-16 15:38:35,131 - Directory['/etc/hadoop'] \{'mode': 0755}

2019-04-16 15:38:35,132 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] \{'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}

2019-04-16 15:38:35,153 - Initializing 6 repositories

2019-04-16 15:38:35,155 - Repository['HDP-2.5'] \{'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', 'mirror_list': None}

2019-04-16 15:38:35,165 - File['/etc/yum.repos.d/HDP.repo'] \{'content': '[HDP-2.5]\nname=HDP-2.5\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,165 - Repository['HDP-UTILS-1.1.0.21'] \{'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}

2019-04-16 15:38:35,169 - File['/etc/yum.repos.d/HDP-UTILS.repo'] \{'content': '[HDP-UTILS-1.1.0.21]\nname=HDP-UTILS-1.1.0.21\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,169 - Repository['ES-Curator-4.x'] \{'base_url': 'http://packages.elastic.co/curator/4/centos/7', 'action': ['create'], 'components': [u'CURATOR', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'CURATOR', 'mirror_list': None}

2019-04-16 15:38:35,172 - File['/etc/yum.repos.d/CURATOR.repo'] \{'content': '[ES-Curator-4.x]\nname=ES-Curator-4.x\nbaseurl=http://packages.elastic.co/curator/4/centos/7\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,172 - Repository['elasticsearch-2.x'] \{'base_url': 'https://packages.elastic.co/elasticsearch/2.x/centos', 'action': ['create'], 'components': [u'ELASTICSEARCH', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ELASTICSEARCH', 'mirror_list': None}

2019-04-16 15:38:35,175 - File['/etc/yum.repos.d/ELASTICSEARCH.repo'] \{'content': '[elasticsearch-2.x]\nname=elasticsearch-2.x\nbaseurl=https://packages.elastic.co/elasticsearch/2.x/centos\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,176 - Repository['kibana-4.x'] \{'base_url': 'http://packages.elastic.co/kibana/4.5/centos', 'action': ['create'], 'components': [u'KIBANA', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'KIBANA', 'mirror_list': None}

2019-04-16 15:38:35,179 - File['/etc/yum.repos.d/KIBANA.repo'] \{'content': '[kibana-4.x]\nname=kibana-4.x\nbaseurl=http://packages.elastic.co/kibana/4.5/centos\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,179 - Repository['METRON-0.4.1'] \{'base_url': 'file:///localrepo', 'action': ['create'], 'components': [u'METRON', 'main'], 'repo_template': '[{{repo_id}}]\nname=\{{repo_id}}\n\{% if mirror_list %}mirrorlist=\{{mirror_list}}\{% else %}baseurl=\{{base_url}}\{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'METRON', 'mirror_list': None}

2019-04-16 15:38:35,182 - File['/etc/yum.repos.d/METRON.repo'] \{'content': '[METRON-0.4.1]\nname=METRON-0.4.1\nbaseurl=file:///localrepo\n\npath=/\nenabled=1\ngpgcheck=0'}

2019-04-16 15:38:35,183 - Package['unzip'] \{'retry_on_repo_unavailability': False, 'retry_count': 5}

2019-04-16 15:38:35,280 - Skipping installation of existing package unzip

2019-04-16 15:38:35,280 - Package['curl'] \{'retry_on_repo_unavailability': False, 'retry_count': 5}

2019-04-16 15:38:35,290 - Skipping installation of existing package curl

2019-04-16 15:38:35,290 - Package['hdp-select'] \{'retry_on_repo_unavailability': False, 'retry_count': 5}

2019-04-16 15:38:35,300 - Skipping installation of existing package hdp-select

2019-04-16 15:38:35,538 - checked_call['rpm -q --queryformat '%\{version}-%\{release}' hdp-select | sed -e 's/\.el[0-9]//g''] \{'stderr': -1}

2019-04-16 15:38:35,590 - checked_call returned (0, '2.5.0.0-1245', '')

2019-04-16 15:38:35,591 - Package['hadoop_2_5_0_0_1245-yarn'] \{'retry_on_repo_unavailability': False, 'retry_count': 5}

2019-04-16 15:38:35,680 - Installing package hadoop_2_5_0_0_1245-yarn ('/usr/bin/yum -d 0 -e 0 -y install hadoop_2_5_0_0_1245-yarn')

2019-04-16 15:38:40,403 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadoop_2_5_0_0_1245-yarn' returned 1. Error: Nothing to do

2019-04-16 15:38:40,403 - Failed to install package hadoop_2_5_0_0_1245-yarn. Executing '/usr/bin/yum clean metadata'

2019-04-16 15:38:40,672 - Retrying to install package hadoop_2_5_0_0_1245-yarn after 30 seconds

2019-04-16 15:39:58,855 - Could not find stack selector for stack: HDP

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)